Using WPA to measure pitcher effectiveness

Last week I wrote about Cubs TV play-by-play man Len Kasper's method of improving the pitching win, and while it wasn't perfect, it better reflected a good pitching performance by essentially removing any decision that wasn't a quality start or in which a pitcher's team didn't score runs. In the middle of the Twitter discussion I had with Len to nail down his definitions, one of my favorite people to follow on Twitter interjected this thought:

@HighHeatStats @LenKasper I'll add that in--thanks!

— Scott Lindholm (@ScottLindholm) May 8, 2014

I liked his angle, and once I figured out how to obtain the data, it was a revelation. If you don't already follow High Heat Stats on Twitter, I suggest you do so right now.

Win Probability Added (WPA) measures how much a given event increases or decreases the odds of a team winning the game. Like every sports statistic, it's a mirror -- what is good for offense is bad for defense and vice versa. For example, in last Friday's game between the Tigers and Red Sox, the only run of the game scored in the top of the first inning when Torii Hunter singled off Jon Lester to drive in Ian Kinsler. This particular play had a WPA of .11, or increased the chances the Tigers would win the game by 11 percentage points. Conversely, it also decreased the chances the Red Sox would win by 11 percentage points as well.

It's important to know what WPA is and isn't -- this is from the FanGraphs definition:

Cumulatively, season-long WPA is not predictive, making it an ineffective number for projections of a player’s talent. However, it is a good describer of what happened in the game and how a win was achieved. And since +1 WPA equals 100% in win expectancy, +1 WPA is the equivalent of one win.

This provides a method to more objectively measure what a pitcher can control -- hits, walks, home runs and the like. This first chart shows the most effective starting pitchers for 2014 through games of Friday, May 16th:

Pitcher	Team	GS	W	L	ND	QS	QS%	WPA	GSc
Johnny Cueto	Reds	9	4	2	3	8	88.9%	2.45	76.7
Max Scherzer	Tigers	9	6	1	2	7	77.8%	2.02	66.2
Mark Buehrle	Blue Jays	8	7	1	0	6	75.0%	1.37	57.6
Masahiro Tanaka	Yankees	8	6	0	2	7	87.5%	1.25	66.6
Jordan Lyles	Rockies	8	5	0	3	6	75.0%	1.22	57.8
Jeff Samardzija	Cubs	9	0	4	5	7	77.8%	1.19	62.3
Julio Teheran	Braves	9	2	3	4	8	88.9%	1.18	62.4
Hisashi Iwakuma	Mariners	3	2	0	1	1	33.3%	1.13	69.3
Zack Greinke	Dodgers	9	7	1	1	5	55.6%	1.12	61.2
Yu Darvish	Rangers	8	3	2	3	6	75.0%	1.12	64.5
Garrett Richards	Angels	8	4	0	4	5	62.5%	1.11	63.0
Adam Wainwright	Cardinals	9	6	2	1	6	66.7%	1.10	65.0
Jose Fernandez	Marlins	8	4	2	2	6	75.0%	1.04	64.0

*** Important note --  only starts are included,  no relief appearances ***

Using Johnny Cueto to explain, he's made nine starts for a record of 4-2 with three no-decisions. Cueto has had eight quality starts so far, implying he's pitched well enough to put the Reds in a position to win. He has a cumulative WPA of 2.45 and his average game score is almost 77. These are all phenomenal numbers, and Cueto is rightly regarded as one of the better pitchers so far this year. What WPA adds is the ability to see the margin between him and the next pitcher, Max Scherzer, as well as the absolute gulf that exists between those two and every other pitcher.

Take note of Hisashi Iwakuma, as he's only made three starts, and his best game by far this season was on May 13th and resulted in a no-decision. Jeff Samardzija's streak of games without a win reached 15 on Friday. It's certainly not his fault as he continues to do what is necessary to keep the Cubs in games, and looking at his WPA helps demonstrate his record isn't due to ineffective pitching on his part.

What pitchers have the lowest WPA so far (minimum five games started)?

Pitcher	Team	GS	W	L	ND	QS	QS%	WPA	GSc
Mike Pelfrey	Twins	5	0	3	2	0	0.0%	-1.40	34.6
CC Sabathia	Yankees	8	3	4	1	2	25.0%	-1.31	46.5
Wandy Rodriguez	Pirates	5	0	2	3	1	20.0%	-1.31	42.4
Kevin Correia	Twins	8	1	5	2	3	37.5%	-1.24	38.1
Danny Salazar	Indians	8	1	4	3	2	25.0%	-1.09	45.3
Clay Buchholz	Red Sox	8	2	3	3	4	50.0%	-1.02	40.3
Wade Miley	D'backs	10	3	4	3	4	40.0%	-0.96	50.4
Bartolo Colon	Mets	8	2	5	1	5	62.5%	-0.92	43.9
Paul Maholm	Dodgers	7	1	4	2	3	42.9%	-0.91	39.1
Erasmo Ramirez	Mariners	6	1	4	1	3	50.0%	-0.87	45.3
Matt Garza	Brewers	8	2	3	3	4	50.0%	-0.86	48.3

The Twins are not getting their $5.5 million out of Mike Pelfrey -- that's a whole lot of negative WPA in only five starts. He's on the DL as I write this and due to come back soon. It could be worse -- they could owe him $71 million over the next three years like the Yankees do CC Sabathia.

Using WPA seems to correlate well with pitching performance so far in 2014, but does it work when used more broadly to measure historical excellence? This chart shows how well pitchers since 1970:

Pitcher	From	To	GS	W	L	ND	QS	QS%	WPA	GScAvg
Roger Clemens	1984	2007	707	354	184	169	465	65.8%	77.73	59.0
Greg Maddux	1986	2008	740	355	226	159	480	64.9%	59.69	56.1
Randy Johnson	1988	2009	603	301	166	136	403	66.8%	52.54	59.9
Pedro Martinez	1992	2009	410	208	97	105	273	66.6%	49.60	61.2
Mike Mussina	1991	2008	536	270	153	113	330	61.6%	40.61	54.8
Roy Halladay	1998	2013	390	200	103	87	257	65.9%	38.09	56.4
Curt Schilling	1988	2007	436	206	134	96	288	66.1%	37.22	58.5
Tom Glavine	1987	2008	682	305	203	174	436	63.9%	35.52	52.7
Kevin Brown	1986	2005	489	213	147	129	319	65.2%	33.52	55.5
Bert Blyleven	1970	1992	685	286	248	151	429	62.6%	31.83	56.9
Tim Hudson	1999	2014	434	209	113	112	281	64.7%	31.26	54.7
Bret Saberhagen	1984	2001	371	162	114	95	224	60.4%	27.91	55.6
Kevin Appier	1989	2004	402	169	136	97	239	59.5%	27.38	53.7
Johan Santana	2000	2012	284	135	76	73	184	64.8%	26.25	58.9
John Smoltz	1988	2009	481	209	149	123	298	62.0%	25.89	56.8
David Cone	1987	2003	419	191	125	103	257	61.3%	25.46	56.8
Jimmy Key	1985	1998	389	179	112	98	237	60.9%	24.79	53.4
Andy Pettitte	1995	2013	521	255	152	114	301	57.8%	24.06	51.7
Roy Oswalt	2001	2013	341	159	99	83	225	66.0%	24.05	55.5

We use numbers and metrics to not just measure something but to see how it compares with others. There's no point in discussing pitcher WPA if we can't see how one pitcher compares to another, as well as see if the results match up with expectations. When the list is headed up by Roger Clemens, Greg Maddux and Randy Johnson, chances are WPA is a measure that can reliably illustrate pitcher effectiveness.

I've just scratched the surface and will return to this topic in the future. I'll end with this chart, showing young pitchers with under 100 starts and introducing a new wrinkle:

Pitcher	From	To	GS	W	L	ND	QS	QS%	WPA	WPA/GS
Jose Fernandez	2013	2014	36	16	8	12	26	72.2%	4.68	.130
Chris Sale	2012	2014	63	31	22	10	45	71.4%	7.38	.117
Hisashi Iwakuma	2012	2014	52	24	10	18	33	63.5%	5.75	.111
Tanner Roark	2013	2014	13	6	2	5	9	69.2%	1.38	.106
Jarred Cosart	2013	2014	18	3	4	11	12	66.7%	1.84	.102
Yordano Ventura	2013	2014	11	2	4	5	5	45.5%	0.96	.087
Sonny Gray	2013	2014	19	10	4	5	14	73.7%	1.41	.074
Yu Darvish	2012	2014	69	32	20	17	45	65.2%	4.91	.071
Hyun-jin Ryu	2013	2014	37	17	10	10	26	70.3%	2.17	.059
Julio Teheran	2011	2014	43	17	12	14	26	60.5%	2.37	.055

WPA per game started is shown in the last column. Fernandez, sadly, is headed for Tommy John surgery, and Chris Sale is slated to return to the White Sox shortly, which I'll believe when I see it. White Sox fans know how good Sale has been, and when he returns from injury, perhaps the rest of baseball will as well. The White Sox issues of the past couple of years can't be blamed on Sale, who has been among the best pitchers in baseball when viewed in terms of WPA.

It will not be easy to integrate WPA into discussions of pitcher effectiveness. A certain comfort level of what odds represent is necessary, and I don't envision a time soon when I'll hear Len Kasper wax poetically about Jeff Samardzija's 1.19 WPA and Jim Deshaies chime in with how that compares with other pitchers. That doesn't mean it isn't useful and worthwhile, because it really does cut through some of the clutter and dramatically shows which pitchers truly are the best in the game.

Data from Baseball-Reference.com. Any errors in compiling and amalgamating the data are the author's. Special thanks to High Heat Stats for the idea.

Scott Lindholm and his wife live in Davenport, IA and have two recent college graduate daughters. Follow him on Twitter @ScottLindholm.

*** Special note for people smarter than me ***

Beyond the Box Score editor Neil Weinberg suggested I use runs above average by the 24 base/out states (RE24) when I tweeted out some of these results last week. I chose WPA because it takes game situation into account and recognizes that runners on first and second in a tie game in the bottom of the seventh inning differs from the same situation in a blowout game. My results wouldn't have changed appreciably if I had used RE24 -- this chart shows the relationship between the two for the approximately 3,800 pitchers who have made starts since 1940:

We also use measures to try to determine what separates good from great, and this graph has the added benefit of showing quite dramatically where the line between really good pitchers and the greatest of all time occurs, somewhere in the career 40 WPA/400 RE24 range.