On the Relative Importance of Plays: Part 1

To some extent, sabermetric research and writing can be split into two broad categories: predicting the future and evaluating the past. In the former category, we have metrics like PECOTA and SIERA and ZiPS, which try to use past performance to project future performance. In the latter, we have WAR and WPA and FIP, which profess to measure what actually happened.

Attempting to predict that which is unpredictable is interesting and fun and gets you jobs in baseball if you're good at it, but I'm not good at it. As a former humanities major at a small liberal arts college, I instead find myself drawn to the more philosophical issues in baseball and sabermetrics. These issues are often encountered when one endeavors to evaluate past performance rather than predict future performance.

One issue is that of measuring the relative importance of plays. We know that some plays are more important to the outcome of a game than others. We even know that the same play will make more of a difference in certain game states than others. Duh. The tough part, the part that has been the object of tons of research and writing over the years, is determining the extent to which a play in one game state matters more or less than a play in another game state.

This is why Win Probability Added (WPA) exists. You, being smart and informed sabermetric citizens, probably know what WPA is, and if you don't, you can probably guess. WPA takes a team's probability of winning before the play (based on of years and years of game data), subtracts it from the team's probability of winning after the play, and assigns that value (and its inverse) to the players involved. It is a very neat and useful metric for certain purposes.

One such purpose is storytelling. That is, while we are watching a game, we can use WPA to determine how much any individual play, or the accumulation of plays involving a certain player, affects the end result of the game. Did that sacrifice bunt help our team win? Which player most increased his team's chances of winning? Which player choked and blew the game? These questions can be answered (imperfectly, I will add) by WPA.

Here's where I get a little strung up. WPA can tell us how much a player improved his team's chances of winning, but that is often mistaken for a player's actual value - or to put it a different way, his importance - in that game.

The best way to illustrate what I mean is through in-game examples:

Situation 1:

Mark Teixeira hits a solo home run with two outs in the bottom of the 1st. Fast forward to the 9th. The score is tied at 1. Robinson Cano comes up, also with two outs and no one on, and hits a walk-off home run.

Teixeira: .104 WPA, .260 WPA/LI
Cano: .465 WPA, .337 WPA/LI

Situation 2:

In the 9th inning, the game is tied. With 2 outs, Teixeira hits a double. Cano then comes up and sends 'em home with a walk-off single.

Teixeira: .0724 WPA, .052 WPA/LI
Cano: .392 WPA, .102 WPA/LI

As one would expect, in both cases, Robinson Cano gets more of the credit for the Yankees winning the game than Teixeira. He got the walk-off hit, and therefore he was the hero. Even if we factor in the leverage index, or the pressure of the situation, Cano still beats out Teixeira by a significant margin.

If you're like me, something about this doesn't seem right. Let's consider Situation 1 first. Both Teixeira and Cano hit a solo home run with two outs in the inning and the game tied. If Cano had not hit his home run, the game would have gone into extra innings. If Teixeira had not hit his home run, all else being equal, the game would have gone into extra innings. Both plays had the same impact on the end result of the game - that is, they gave the Yankees one run.

Situation 2 is a bit trickier because Teixeira and Cano do not come up with the same base-out state. Taken out of context, Teixeira's double is obviously a better outcome than Cano's single. But with two outs and no one on base, a double is relatively unhelpful compared to a single with a man on second.

Think of it this way. Teixeira's double increases the Yankees' chances of scoring a run (which would win the game) by 15.5%, yet a home run would have increased their chances by 92.5%. On the other hand - and for the sake of argument let's assume a single guarantees that the runner on 2nd will score - Cano only has to hit a single. A double, triple, and home run would do no more to increase the chances of scoring than would a single.

Nevertheless, there's something bothersome about the fact that Teixeira gets so little credit for his double. After all, if he had not hit the double, Cano would not have even had the chance to end the game - it would have been over already. Sure, the double did less to improve their chances of winning than Cano's single, but the only other outcome that would have been significantly better for Teixeira was a home run.

The common problem that WPA faces in these two examples is that of uncertainty. At the time of Teixeira's first inning home run, we have no idea how much of a difference it will make in the end. Similarly, we don't know if Teixeira's double in the 9th will make a difference at the time of its occurrence. As it turns out, both plays were crucial to the end result of the game, and when we look back on the game, we can see that. WPA is measured at the time of the play, but if we want to truly determine the relative importance of plays, we need to be able to look at the whole picture.

For now, I don't have a concrete solution to these issues. However, I hope to have illustrated a few problems with using WPA, or even WPA/LI, to evaluate a player's contribution to his team. Next time, I will consider some pre-existing potential solutions, and maybe I'll even come up with some ideas of my own.

Credit to the sweet WPA calculator at THT, Tango's RE24 matrix, and a great discussion on the subject over on The Book Blog