Filed under:

# Using WPA to measure pitcher effectiveness

Can win probability added (WPA) be used to measure starting pitcher effectiveness?

Last week I wrote about Cubs TV play-by-play man Len Kasper's method of improving the pitching win, and while it wasn't perfect, it better reflected a good pitching performance by essentially removing any decision that wasn't a quality start or in which a pitcher's team didn't score runs. In the middle of the Twitter discussion I had with Len to nail down his definitions, one of my favorite people to follow on Twitter interjected this thought:

I liked his angle, and once I figured out how to obtain the data, it was a revelation. If you don't already follow High Heat Stats on Twitter, I suggest you do so right now.

Win Probability Added (WPA) measures how much a given event increases or decreases the odds of a team winning the game. Like every sports statistic, it's a mirror -- what is good for offense is bad for defense and vice versa. For example, in last Friday's game between the Tigers and Red Sox, the only run of the game scored in the top of the first inning when Torii Hunter singled off Jon Lester to drive in Ian Kinsler. This particular play had a WPA of .11, or increased the chances the Tigers would win the game by 11 percentage points. Conversely, it also decreased the chances the Red Sox would win by 11 percentage points as well.

It's important to know what WPA is and isn't -- this is from the FanGraphs definition:

Cumulatively, season-long WPA is not predictive, making it an ineffective number for projections of a player’s talent. However, it is a good describer of what happened in the game and how a win was achieved. And since +1 WPA equals 100% in win expectancy, +1 WPA is the equivalent of one win.

This provides a method to more objectively measure what a pitcher can control -- hits, walks, home runs and the like. This first chart shows the most effective starting pitchers for 2014 through games of Friday, May 16th:

Pitcher Team GS
L
ND QS QS% WPA GSc
Johnny Cueto Reds 9 4 2 3 8 88.9% 2.45 76.7
Max Scherzer Tigers 9 6 1 2 7 77.8% 2.02 66.2
Mark Buehrle Blue Jays 8 7 1 0 6 75.0% 1.37 57.6
Masahiro Tanaka Yankees 8 6 0 2 7 87.5% 1.25 66.6
Jordan Lyles Rockies 8 5 0 3 6 75.0% 1.22 57.8
Jeff Samardzija Cubs 9 0 4 5 7 77.8% 1.19 62.3
Julio Teheran Braves 9 2 3 4 8 88.9% 1.18 62.4
Hisashi Iwakuma Mariners 3 2 0 1 1 33.3% 1.13 69.3
Zack Greinke Dodgers 9 7 1 1 5 55.6% 1.12 61.2
Yu Darvish Rangers 8 3 2 3 6 75.0% 1.12 64.5
Garrett Richards Angels 8 4 0 4 5 62.5% 1.11 63.0
Adam Wainwright Cardinals 9 6 2 1 6 66.7% 1.10 65.0
Jose Fernandez Marlins 8 4 2 2 6 75.0% 1.04 64.0

*** Important note --  only starts are included,  no relief appearances ***

Using Johnny Cueto to explain, he's made nine starts for a record of 4-2 with three no-decisions. Cueto has had eight quality starts so far, implying he's pitched well enough to put the Reds in a position to win. He has a cumulative WPA of 2.45 and his average game score is almost 77. These are all phenomenal numbers, and Cueto is rightly regarded as one of the better pitchers so far this year. What WPA adds is the ability to see the margin between him and the next pitcher, Max Scherzer, as well as the absolute gulf that exists between those two and every other pitcher.

Take note of Hisashi Iwakuma, as he's only made three starts, and his best game by far this season was on May 13th and resulted in a no-decision. Jeff Samardzija's streak of games without a win reached 15 on Friday. It's certainly not his fault as he continues to do what is necessary to keep the Cubs in games, and looking at his WPA helps demonstrate his record isn't due to ineffective pitching on his part.

What pitchers have the lowest WPA so far (minimum five games started)?

Pitcher Team GS
L
ND QS QS% WPA GSc
Mike Pelfrey Twins 5 0 3 2 0 0.0% -1.40 34.6
CC Sabathia Yankees 8 3 4 1 2 25.0% -1.31 46.5
Wandy Rodriguez Pirates 5 0 2 3 1 20.0% -1.31 42.4
Kevin Correia Twins 8 1 5 2 3 37.5% -1.24 38.1
Danny Salazar Indians 8 1 4 3 2 25.0% -1.09 45.3
Clay Buchholz Red Sox 8 2 3 3 4 50.0% -1.02 40.3
Wade Miley D'backs 10 3 4 3 4 40.0% -0.96 50.4
Bartolo Colon Mets 8 2 5 1 5 62.5% -0.92 43.9
Paul Maholm Dodgers 7 1 4 2 3 42.9% -0.91 39.1
Erasmo Ramirez Mariners 6 1 4 1 3 50.0% -0.87 45.3
Matt Garza Brewers 8 2 3 3 4 50.0% -0.86 48.3

The Twins are not getting their \$5.5 million out of Mike Pelfrey -- that's a whole lot of negative WPA in only five starts. He's on the DL as I write this and due to come back soon. It could be worse -- they could owe him \$71 million over the next three years like the Yankees do CC Sabathia.

Using WPA seems to correlate well with pitching performance so far in 2014, but does it work when used more broadly to measure historical excellence? This chart shows how well pitchers since 1970:

Pitcher From To GS W L ND QS QS% WPA GScAvg
Roger Clemens 1984 2007 707 354 184 169 465 65.8% 77.73 59.0
Greg Maddux 1986 2008 740 355 226 159 480 64.9% 59.69 56.1
Randy Johnson 1988 2009 603 301 166 136 403 66.8% 52.54 59.9
Pedro Martinez 1992 2009 410 208 97 105 273 66.6% 49.60 61.2
Mike Mussina 1991 2008 536 270 153 113 330 61.6% 40.61 54.8
Roy Halladay 1998 2013 390 200 103 87 257 65.9% 38.09 56.4
Curt Schilling 1988 2007 436 206 134 96 288 66.1% 37.22 58.5
Tom Glavine 1987 2008 682 305 203 174 436 63.9% 35.52 52.7
Kevin Brown 1986 2005 489 213 147 129 319 65.2% 33.52 55.5
Bert Blyleven 1970 1992 685 286 248 151 429 62.6% 31.83 56.9
Tim Hudson 1999 2014 434 209 113 112 281 64.7% 31.26 54.7
Bret Saberhagen 1984 2001 371 162 114 95 224 60.4% 27.91 55.6
Kevin Appier 1989 2004 402 169 136 97 239 59.5% 27.38 53.7
Johan Santana 2000 2012 284 135 76 73 184 64.8% 26.25 58.9
John Smoltz 1988 2009 481 209 149 123 298 62.0% 25.89 56.8
David Cone 1987 2003 419 191 125 103 257 61.3% 25.46 56.8
Jimmy Key 1985 1998 389 179 112 98 237 60.9% 24.79 53.4
Andy Pettitte 1995 2013 521 255 152 114 301 57.8% 24.06 51.7
Roy Oswalt 2001 2013 341 159 99 83 225 66.0% 24.05 55.5

We use numbers and metrics to not just measure something but to see how it compares with others. There's no point in discussing pitcher WPA if we can't see how one pitcher compares to another, as well as see if the results match up with expectations. When the list is headed up by Roger Clemens, Greg Maddux and Randy Johnson, chances are WPA is a measure that can reliably illustrate pitcher effectiveness.

I've just scratched the surface and will return to this topic in the future. I'll end with this chart, showing young pitchers with under 100 starts and introducing a new wrinkle:

Pitcher From To GS W L ND QS QS% WPA WPA/GS
Jose Fernandez 2013 2014 36 16 8 12 26 72.2% 4.68 .130
Chris Sale 2012 2014 63 31 22 10 45 71.4% 7.38 .117
Hisashi Iwakuma 2012 2014 52 24 10 18 33 63.5% 5.75 .111
Tanner Roark 2013 2014 13 6 2 5 9 69.2% 1.38 .106
Jarred Cosart 2013 2014 18 3 4 11 12 66.7% 1.84 .102
Yordano Ventura 2013 2014 11 2 4 5 5 45.5% 0.96 .087
Sonny Gray 2013 2014 19 10 4 5 14 73.7% 1.41 .074
Yu Darvish 2012 2014 69 32 20 17 45 65.2% 4.91 .071
Hyun-jin Ryu 2013 2014 37 17 10 10 26 70.3% 2.17 .059
Julio Teheran 2011 2014 43 17 12 14 26 60.5% 2.37 .055

WPA per game started is shown in the last column. Fernandez, sadly, is headed for Tommy John surgery, and Chris Sale is slated to return to the White Sox shortly, which I'll believe when I see it. White Sox fans know how good Sale has been, and when he returns from injury, perhaps the rest of baseball will as well. The White Sox issues of the past couple of years can't be blamed on Sale, who has been among the best pitchers in baseball when viewed in terms of WPA.

It will not be easy to integrate WPA into discussions of pitcher effectiveness. A certain comfort level of what odds represent is necessary, and I don't envision a time soon when I'll hear Len Kasper wax poetically about Jeff Samardzija's 1.19 WPA and Jim Deshaies chime in with how that compares with other pitchers. That doesn't mean it isn't useful and worthwhile, because it really does cut through some of the clutter and dramatically shows which pitchers truly are the best in the game.

Data from Baseball-Reference.com. Any errors in compiling and amalgamating the data are the author's. Special thanks to High Heat Stats for the idea.

Scott Lindholm and his wife live in Davenport, IA and have two recent college graduate daughters. Follow him on Twitter @ScottLindholm.

*** Special note for people smarter than me ***

Beyond the Box Score editor Neil Weinberg suggested I use runs above average by the 24 base/out states (RE24) when I tweeted out some of these results last week. I chose WPA because it takes game situation into account and recognizes that runners on first and second in a tie game in the bottom of the seventh inning differs from the same situation in a blowout game. My results wouldn't have changed appreciably if I had used RE24 -- this chart shows the relationship between the two for the approximately 3,800 pitchers who have made starts since 1940:

We also use measures to try to determine what separates good from great, and this graph has the added benefit of showing quite dramatically where the line between really good pitchers and the greatest of all time occurs, somewhere in the career 40 WPA/400 RE24 range.