How value in closers is found outside WAR

When Jose Reyes left the Mets after his 2011 batting champion season, the Mets won three fewer games the following season. When Albert Pujols shockingly left St. Louis to play in Anaheim, the Cardinals lost two wins the next year, while the Angels added three. Using WAR as a baseline, Pujols added 3.7 wins, while Reyes added 4.1 wins in their respective debut seasons for their new teams. But what difference did each player really make in the win-loss column? How many more games would have the 2012 Mets won had Reyes stayed with the team? Did the Angels improve because they added Pujols?

General managers face an inexact science when trying to evaluate how much value an individual player can add to a team sport. Advanced statistics have made their jobs easier. WAR is a widely used metric that uses run statistics to estimate the number of wins a player adds to their team beyond a replacement level player. While WAR is helpful, it is still difficult to pinpoint exactly how many wins a player adds to a ball club in the standings.

Which brings us to closers. Perhaps the most defined role in baseball, closers are expected to get the final three outs of a game, preferably in situations when their team has a lead and the score is within three runs. More than any other position, we can evaluate the impact of a closer on a team's win total by looking at the number of games they fail to do their job, and ultimately, cost their team a win.

Let me make clear that this article is focused on outcomes. Statistics such as fWAR, which relies heavily on FIP, which evaluates things within a pitcher's control - such as strikeouts and home runs allowed - are important in evaluating how a pitcher does their job, and importantly, potentially predict how they will do it in the future. Most pitching statistics are focused on the how.

The how is important to understand, but the question GMs needs to ask themselves when evaluating a closer is what is their end game? Pun intended. A closer is paid to get the final three outs of a ballgame. How he gets those outs matters less than if he gets them at all. Market value is exaggerated by how closers get outs instead of how successful they are in performing their defined role of preserving a lead and/or preventing losses from bad outings. Team executives could better understand the impact closers really have on their team's win-loss records by looking at a statistic on the opposite spectrum of wins added, and that is, losses added.

Before deriving loses added, let's understand the components that are important to pay attention to when evaluating a closer. Obviously, there are saves. The most basic statistic that has essentially sparked the role of the closer. A more telling statistic, introduced by FanGraphs, is meltdowns. In order to give more weight to reliever performance outside of traditional saves, meltdowns, and its companion shutdowns, are important statistics. The idea is to value how much a reliever contributes towards a team's odds of winning a game by counting the number of shutdowns (positive win probability added of 6%) versus meltdowns (negative win probability of 6%).

Using meltdowns instead of just blown saves to measure how often a closer fails to do their job allows us to include non-save situation appearances that are still crucial. For instance, a tie game in the 8th, failing to get a key out. Or with the team trailing 2-1, and then allowing four more runs to put the game out of reach. Of course, in any of these situations, when the game doesn't end fatally, as in the case with a blown save in the walk-off form, there is no guarantee that the reliever's performance directly resulted in the team losing. The point is that it still had a negative impact that greatly reduced the probability of the team winning.

In fact, over the past three seasons, when a reliever melts down during a game, his team loses 75% of the time. So if we take a reliever's meltdowns, factoring in the propensity those meltdowns turn into losses, we can get a losses added value. To illustrate this concept, Craig Kimbrel had five meltdowns in 2013. At the .752 loss rate, those five meltdowns would result in 3.6 losses. Since we want to compare closers who face different opportunities based on their team success - better teams have more save or shutdown opportunities - we need to scale Kimbrel's meltdowns to a comparable level. Using his SD/MD rate of 89% to adjust his total to a per 50 opportunity value, Kimbrel added 4.3 losses in 2013.

MLB winning percentage for teams when reliever has meltdown

What does this all tell us? Last season, Craig Kimbrel had 50 saves, disposing opposing hitters in the process to the tune of a 13.16 K/9 rate and a minuscule FIP of 1.93. But how much different would the Braves have fared with Kimbrel as their closer instead of somebody else?

Let's compare him to Brandon League. League performed to a -1.0 WAR in 2013, which was 3.2 fewer wins added than Craig Kimbrel. By looking at the two from a losses added perspective, fWAR understates the difference in actual wins gained by each of their teams. On a 50 chance scale, League would have added 18 losses through his meltdowns compared to Kimbrel's 4.3, or a gaping 13.7 win difference!

General managers are getting better at using advanced statistics, like WAR, in evaluating contracts for their players. Closers offer a unique opportunity to evaluate players based truly on their impact to the standings. Perhaps WAR doesn't tell the picture so clearly for 9th-inning hurlers. Use the dashboard below to identify which closers are being valued at a WAR discount or premium. For example, in 2013, Chris Perez pitched to a -0.9 WAR, nearly two wins below Jim Johnson. Yet, given the same opportunities, Perez would have created 0.3 fewer losses than Johnson.

<script type='text/javascript' src='http://public.tableausoftware.com/javascripts/api/viz_v1.js'></script><div class='tableauPlaceholder' style='width: 544px; height: 789px;'><noscript><a href='#'><img alt='Dashboard 1 ' src='http:&#47;&#47;public.tableausoftware.com&#47;static&#47;images&#47;Cl&#47;Closers&#47;Dashboard1&#47;1_rss.png' style='border: none' /></a></noscript><object class='tableauViz' width='544' height='789' style='display:none;'><param name='host_url' value='http%3A%2F%2Fpublic.tableausoftware.com%2F' /> <param name='site_root' value='' /><param name='name' value='Closers&#47;Dashboard1' /><param name='tabs' value='no' /><param name='toolbar' value='yes' /><param name='static_image' value='http:&#47;&#47;public.tableausoftware.com&#47;static&#47;images&#47;Cl&#47;Closers&#47;Dashboard1&#47;1.png' /> <param name='animate_transition' value='yes' /><param name='display_static_image' value='yes' /><param name='display_spinner' value='yes' /><param name='display_overlay' value='yes' /><param name='display_count' value='yes' /></object></div><div style='width:544px;height:22px;padding:0px 10px 0px 0px;color:black;font:normal 8pt verdana,helvetica,arial,sans-serif;'><div style='float:right; padding-right:8px;'><a href='http://www.tableausoftware.com/public/about-tableau-products?ref=http://public.tableausoftware.com/views/Closers/Dashboard1' target='_blank'>Learn About Tableau</a></div></div>

. . .

All statistics courtesy of Baseball-Reference.

Jeffrey Bellone is a writer and editor at Beyond The Box Score and can also be found writing about the Mets at Amazin' Avenue. You can follow and interact with him on Twitter @OverWhitestone.