The Braves Off Season - Playoff Probability Edition
The Atlanta Braves have had a busy (and sometimes controversial) offseason as they revamped their bullpen, subtracted an arm from their surplus of starters, and picked up a few average-ish bats. The individual moves have been covered in depth by our outstanding group of writers here at BtB (Vasquez, Soriano) and by some of the heavy hitters in the sabr community (Cameron, Tango). My goal in this article is to wrap all of the moves together and examine the offense as a whole. My metric of choice is playoff probability added/subtracted, and I am going to look at two separate approaches to estimate the gains/losses. First I will leverage the work Nick did here that looked at historic playoff probabilities given certain true talent level wins. Second I will take a more contextual look at the problem by seeing how the moves affected the division race as it currently stack ups.
Background Work
Critical to both approaches is an estimate of the Braves true talent level in terms of wins both before the moves and after the moves. I used current CHONE projections for offense and pitching, my own defensive projections, and playing time estimates based on the Fangraphs fan’s projections to come up with WAR projections. Using this method I estimated 92.8 wins before the moves and 89.4 wins after them.
The results are after the jump
First Approach
First I’ll simply reprint Nick’s charts as points of reference (if you want more details check out the linked article).
Clearly the one of use to us is the third order wins chart. We can simply plug the estimated wins into the regression equation (which is embedded in the spreadsheets linked in the description article) and calculate the difference in probabilities.
P(Before) = 0.85
P(After) = 0.70
Clearly we can all do the math and see that this method says the Braves gave up 0.15 of playoff probability THIS YEAR with their moves. Also of note is that it also says they still have 0.70 chance to get in.
Second Approach
The inquisitive analyst / skeptic may say, "But Steve, why are we using historical information when it doesn’t specifically take into account division strength? Clearly we know that the Braves have to contend with the Phillies, not to mention the Marlins, etc."
My answer would be that it’s tough to draw too many conclusions about division strength because there are enough rosters still in flux. However with the Phillies being set at so many positions I think it could be worthwhile to compare just those two teams. In order to do that, I need a similar estimate of true team talent for the Phils. Going through a similar process to the one outlined above yields 94.8 wins. So the question then becomes, "How often will either a 92.8 or 89.4 win team finish ahead of a 94.8 win team, and what is the resulting loss of going from ~93 to ~89 wins. To answer that question I’m going to make some fairly large assumptions. I’m going to assume the number of actual wins is normally distributed with a mean of the true talent wins (TTW) and a standard deviation of 5 wins (this is more or less just an educated guess using a few years a data and looking at the SD). With these assumptions in place I can answer the posed question by simply looking at the statistic of Actual Braves Wins- Acutal Phillies Wins (henceforth ActB-ActP) and calculating P(B-P)>0. Based on the assumption of normality (ActB-ActP)~N(TTW_B-TTW_P,sqrt(50)) [which is statspeak for the new statistic is also a normal distribution with a mean that is the difference of the two TTW and standard deviation that is sqrt(variance of TTW_B + variance of TTW_P)]. Since the new statistic is also normal, finding various probabilities just became much easier. Plugging and chugging into the normal distribution in excel yields
P(Braves (Before)>Phillies) = 0.39
P(Braves (After)>Phillies )= 0.22
Again we can all do the math and see that the numbers come to a difference of 0.17
Odds and Ends
An obvious question is, "What happens if the Braves spend more of the money they saved and spend it on another player?" Without too much detail, adding one WAR would put them at 0.75 to make the playoffs (method 1).
When I mentioned to Nick that I'd be using his curves in this piece, he suggested that I redo the regression using projected wins as the variable instead of 3rd order wins. This is definitely the correct way to go about it. However I think if I took the actual wins curve and applied a predicted to actual transformation I think I'd end up with something that looks a lot like the 3rd order wins curve (the transformation would likely just increase the standard deviation).
Clearly up until this point the analysis has been focused on this year only, which is not the ideal way to look at the trade. Clearly there is value to the moves (notably the Vasquez deal) that the Braves will get in ensuing seasons. It's a little trickier to asses the future years using a similar metric, as it's hard to characterize the value of the payroll flexibility gained by picking up and average cost controlled outfielder and a prospect. The question is whether the 10-15% that the Braves lost this year in playoff probability can be gained back in the upcoming seasons because of having to purchase less WAR on the open market. Someone smarter than I am is going to have to attack that question. My gut says it'll all come pretty close to even in the end, but that could be Christmas leftovers just disagreeing with me.
1 recs |
15 comments
|
Comments
Great stuff Steve.
I really like Playoff Probability analysis. With an estimate as to how much the playoffs are worth to a team, we can also find out how much these moves are costing in terms of value along with production. From what I recall, it’s $20M for a playoff spot (someone can and should correct me if I’m wrong). That means the Braves might have cost themselves an additional $3-3.5M this season in terms of playoff odds. But as you said, the odds may increase in subsequent years, so that analysis isn’t entirely complete.
Marlin Maniac, a Florida Marlins blog
Come attend Intro to Sabermetrics 101!
Check me out at Beyond the Box Score as well.
Thanks
And I like the thought of applying dollar values, if nothing else it’s a point of comparison.
I might make a little series out of this. Examine some more of the teams that had a busy off season (Seattle, Philly, whoever lands Holliday, Boston, etc.). I’ll see what kind of reaction this gets and go from there.
by stevesommer05 on Dec 28, 2009 10:36 AM EST up reply actions
Hi, I’m a rabid Braves fan and a Sabermetrics neophyte, so I love posts like this one. I’m curious (if you don’t mind) for a bit more of the behind the scenes info on what caused the Braves 2010 win expectancy to go down. I assume most of the lost projected wins for 2010 came from losing Vazquez, but would love to see a player-by-player breakdown of what the Braves added and lost.
Also, to the other beleaguered Braves fans out there I think it’s worth noting that after the salary relief from the Vazquez trades the Braves may have in the neighborhood of 7 to 9 million left to spend (according to Braves beat writer DOB). Now it’s probably a stretch to think the Braves could add 3.4 more wins with this money, but things may get a little better before the offseason is over.
Thanks for stopping by
Click on the hyper-link behind the “WAR projections” in the background section. That should take you to a google docs spreadsheet that has a breakdown for the two Braves scenarios and the Phillies as well. It’s WAR by player, so I think it’ll answer at least some of your questions. If you’re still a little fuzzy after checking that out, come on back and ask more questions.
by stevesommer05 on Dec 28, 2009 11:20 AM EST up reply actions
Excellent
Thanks for posting this. I really appreciate it.
The bad news is they already spent the “savings” of the Vazquez trade. They jumped early on two free agent relievers (Wagner (7.5) + Saito (5)).
Hudson: offseason move or not?
Thanks! That’s exactly what I was looking for. I feel a little silly for not finding it the first time through the article. I guess I’m guilty of sometimes passing over the hyperlinks a little too quickly.
I was curious to see how you handled the whole 6 starting pitcher debacle. How did you decide on where to count Hudson? From your spreadsheet it looks like you counted Hudson as part of both the pre- and post- offseason Braves teams. The Braves had a 2010 option on him, but they instead signed him to a 3 year contract. Should Hudson be considered one of the Braves offseason moves (after all, if they don’t resign him or accept his option, the Vazquez trade probably never goes down)? I realize this change wouldn’t affect how good the Braves are projected to be in your “Braves After” spreadsheet, but it would change your projected wins in the “Braves Before” spreadsheet. Under this scenario the “Braves Before” would have Kawakami in the starting rotation as a replacement for Hudson and Kawakami would be replaced in the bullpen, where I assume he is currently being factored, resulting in (I think) less wins for the “Braves Before”. I hope this is getting at a substantive difference, because I am certainly not trying quibble with you over semantics. What do you think?
Hmmm good call
I did not include Hudson in the set of moves. You are right, that would bring the before picture downward, and probably put it more in-line with the after. If I get a few free minutes I’ll see what kind of impact it has.
by stevesommer05 on Dec 28, 2009 1:18 PM EST up reply actions
Along these lines
When you used the fans’ projected playing times, did you adjust them so the PAs and IP totaled up right in both the before and after projections, or just use the straight fan projections for playing time? 92.8 wins seems like a pretty high projection for Atlanta pre-trade, but maybe it’s not.
I did some adjustments
based on some numbers that Sky had for tagets in his team WAR spreadsheet from last year. Now that I think about it though I did PAs wrong because I included pitchers and his targets didn’t. I thought the before numbers were a little high also… FWIW
by stevesommer05 on Dec 28, 2009 7:10 PM EST up reply actions
When I was working on the PPA article, I got the idea for a logistic regression from an article by Nate Silver
http://www.baseballprospectus.com/article.php?articleid=4618
Really a fantastic article, too bad it’s behind the paywall.
At any rate, Nate mentioned that he used a logistic regression find the playoff curve of regular season wins, looked at the RMSE with projected wins and regular season wins, then used Bayesian inference to combine the two. I have no idea how he did it.
Aslo
That thing with the normal distribution is similar the same thing that I did for my playoff odds earlier this year.
http://www.beyondtheboxscore.com/2009/7/16/931675/btb-playoff-odds-july-16th
Plugging estimate pW% into my spreadsheet, using .500 for the Marlins, .530 for the Mets, .450 for the Nationals .573 (92.8/162) for the Braves and .585 for the Phillies, I get 36% odds for the Braves to win the division. With a .552 pW%, 89.4 wins, I get 25% odds.
Good stuff anyways. I love articles having to do with playoff probability.
minor suggestion
5 wins is probably a bit low for the SD. Assuming you had a perfect projection for the mean number of wins, the SD would still be about 6 wins (SD=sqrt(162*w%*(1-w%)), which is about 6 for teams between .400 and .600), so when you add the uncertainty of the projection, the SD will probably be a bit higher than 6. It’s not a huge deal, but it makes a difference of a few percentage points in the second method in comparing the Braves to just the Phillies.
Thanks
Like there was a lot of hand waving going on to come up with the 5 number.
by stevesommer05 on Dec 28, 2009 7:11 PM EST up reply actions

by 





























