How value in closers is found outside of WAR

Kirby Lee-USA TODAY Sports

Do closers present the most clear picture of wins added (or lost)?

When Jose Reyes left the Mets after his 2011 batting champion season, the Mets won three fewer games the following season. When Albert Pujols shockingly left St. Louis to play in Anaheim, the Cardinals lost two wins the next year, while the Angels added three. Using WAR as a baseline, Pujols added 3.7 wins, while Reyes added 4.1 wins in their respective debut seasons for their new teams. But what difference did each player really make in the win-loss column? How many more games would have the 2012 Mets won had Reyes stayed with the team? Did the Angels improve because they added Pujols?

General managers face an inexact science when trying to evaluate how much value an individual player can add to a team sport. Advanced statistics have made their jobs easier. WAR is a widely used metric that uses run statistics to estimate the number of wins a player adds to their team beyond a replacement level player. While WAR is helpful, it is still difficult to pinpoint exactly how many wins a player adds to a ball club in the standings.

Which brings us to closers. Perhaps the most defined role in baseball, closers are expected to get the final three outs of a game, preferably in situations when their team has a lead and the score is within three runs. More than any other position, we can evaluate the impact of a closer on a team's win total by looking at the number of games they fail to do their job, and ultimately, cost their team a win.

Let me make clear that this article is focused on outcomes. Statistics such as fWAR, which relies heavily on FIP, which evaluates things within a pitcher's control - such as strikeouts and home runs allowed - are important in evaluating how a pitcher does their job, and importantly, potentially predict how they will do it in the future. Most pitching statistics are focused on the how.

The how is important to understand, but the question GMs needs to ask themselves when evaluating a closer is what is their end game? Pun intended. A closer is paid to get the final three outs of a ballgame. How he gets those outs matters less than if he gets them at all. Market value is exaggerated by how closers get outs instead of how successful they are in performing their defined role of preserving a lead and/or preventing losses from bad outings. Team executives could better understand the impact closers really have on their team's win-loss records by looking at a statistic on the opposite spectrum of wins added, and that is, losses added.

Before deriving loses added, let's understand the components that are important to pay attention to when evaluating a closer. Obviously, there are saves. The most basic statistic that has essentially sparked the role of the closer. A more telling statistic, introduced by FanGraphs, is meltdowns. In order to give more weight to reliever performance outside of traditional saves, meltdowns, and its companion shutdowns, are important statistics. The idea is to value how much a reliever contributes towards a team's odds of winning a game by counting the number of shutdowns (positive win probability added of 6%) versus meltdowns (negative win probability of 6%).

Using meltdowns instead of just blown saves to measure how often a closer fails to do their job allows us to include non-save situation appearances that are still crucial. For instance, a tie game in the 8th, failing to get a key out. Or with the team trailing 2-1, and then allowing four more runs to put the game out of reach. Of course, in any of these situations, when the game doesn't end fatally, as in the case with a blown save in the walk-off form, there is no guarantee that the reliever's performance directly resulted in the team losing. The point is that it still had a negative impact that greatly reduced the probability of the team winning.

In fact, over the past three seasons, when a reliever melts down during a game, his team loses 75% of the time. So if we take a reliever's meltdowns, factoring in the propensity those meltdowns turn into losses, we can get a losses added value. To illustrate this concept, Craig Kimbrel had five meltdowns in 2013. At the .752 loss rate, those five meltdowns would result in 3.6 losses. Since we want to compare closers who face different opportunities based on their team success - better teams have more save or shutdown opportunities - we need to scale Kimbrel's meltdowns to a comparable level. Using his SD/MD rate of 89% to adjust his total to a per 50 opportunity value, Kimbrel added 4.3 losses in 2013.

MLB winning percentage for teams when reliever has meltdown

Season W L Win %
2011 484 1532 0.240
2012 553 1493 0.270
2013 511 1661 0.235

What does this all tell us? Last season, Craig Kimbrel had 50 saves, disposing opposing hitters in the process to the tune of a 13.16 K/9 rate and a minuscule FIP of 1.93. But how much different would the Braves have fared with Kimbrel as their closer instead of somebody else?

Let's compare him to Brandon League. League performed to a -1.0 WAR in 2013, which was 3.2 fewer wins added than Craig Kimbrel. By looking at the two from a losses added perspective, fWAR understates the difference in actual wins gained by each of their teams. On a 50 chance scale, League would have added 18 losses through his meltdowns compared to Kimbrel's 4.3, or a gaping 13.7 win difference!

General managers are getting better at using advanced statistics, like WAR, in evaluating contracts for their players. Closers offer a unique opportunity to evaluate players based truly on their impact to the standings. Perhaps WAR doesn't tell the picture so clearly for 9th-inning hurlers. Use the dashboard below to identify which closers are being valued at a WAR discount or premium. For example, in 2013, Chris Perez pitched to a -0.9 WAR, nearly two wins below Jim Johnson. Yet, given the same opportunities, Perez would have created 0.3 fewer losses than Johnson.

. . .

All statistics courtesy of Baseball-Reference.

Jeffrey Bellone is a writer and editor at Beyond The Box Score and can also be found writing about the Mets at Amazin' Avenue. You can follow and interact with him on Twitter @OverWhitestone.

Log In Sign Up

Log In Sign Up

Please choose a new SB Nation username and password

As part of the new SB Nation launch, prior users will need to choose a permanent username, along with a new password.

Your username will be used to login to SB Nation going forward.

I already have a Vox Media account!

Verify Vox Media account

Please login to your Vox Media account. This account will be linked to your previously existing Eater account.

Please choose a new SB Nation username and password

As part of the new SB Nation launch, prior MT authors will need to choose a new username and password.

Your username will be used to login to SB Nation going forward.

Forgot password?

We'll email you a reset link.

If you signed up using a 3rd party account like Facebook or Twitter, please login with it instead.

Forgot password?

Try another email?

Almost done,

By becoming a registered user, you are also agreeing to our Terms and confirming that you have read our Privacy Policy.

Join Beyond the Box Score

You must be a member of Beyond the Box Score to participate.

We have our own Community Guidelines at Beyond the Box Score. You should read them.

Join Beyond the Box Score

You must be a member of Beyond the Box Score to participate.

We have our own Community Guidelines at Beyond the Box Score. You should read them.




Choose an available username to complete sign up.

In order to provide our users with a better overall experience, we ask for more information from Facebook when using it to login so that we can learn more about our audience and provide you with the best possible experience. We do not store specific user data and the sharing of it is not required to login with Facebook.