Daily Box Score 10/9: The Fog of Playoffs
The playoffs are underway, and there have already been some exciting moments. The old refrain that the playoffs are a crapshoot is as tired as it is common (even in its more expletive-peppered variety).
Forgive me for stating the obvious, but in a straight casino, a crapshoot really is random. The odds may have been jiggered so that a savvy craps player has a 1-2% disadvantage against the house, but the dice are not weighted. The outcomes truly are random.
In the playoffs, we all know that isn't true. But we also realize that the best team does not win 100% of the time. So how can we fix a point somewhere in the middle?
Table of Contents
Single Game
Series
Other Resources
Discussion Question of the Day
The hardest situation to predict in a baseball game is the single plate appearance. As you start to aggregate plate appearances, things get progressively easier. By the time you get to a full season, projections start to look better and better (but this, too, remains difficult).
Somewhere in the middle is the difficulty of predicting a single game. Consider that the Yankees, the team with the highest winning percentage in the majors this year, won fewer than 64% of its games. Assuming that is the true estimation of team strength (we'll get to that in a minute), you'd be wrong picking the Yankees against an average team more than once every three times.
Now consider that the Yankees in the playoffs face above average teams (insert joke about Twins' mediocrity here). So how can we sort out how well playoff teams will do against each other?
Because this is sort of a do-it-yourself website that entertains the shop-table tinkering of people who are inclined to ignore warnings not to try this at home, I thought I would walk us through the math.
The classic way to solve a problem like this is to use the log5 method, which was developed by Bill James in his 1981 Abstract. The log5 method can be written in algebraic form like this:
Where "A" is the winning percentage of one team and "B" is the winning percentage of the other team. If you'd like more detailed information on how this formula can be derived, see the August, 2004 issue of SABR's "By the Numbers," which Phil Birnbaum has helpfully archived here.
You can actually use any number here you like; it doesn't have to be the backward-looking actual season record. I typically use a teams third-order winning percentage (see here), but you could use a team's total WAR to imply a winning percentage. You could also, as JinAZ does here, use his wonderful TQI. Today, to keep it in-house, I'll use JinAZ's numbers. He has graciously provided me with the end of season numbers for the Rox and Phils.
Let's put the method to work to find the probability that my beloved Phillies will win tomorrow evening. At the end of the season, Philadelphia had a TQI of .525. Colorado's was .572.
Plugging that in, we get:
WPct = ( (.525 - .525 * .572) / (.525 + .572 - 2 * .525 * .572) )
Solving,
WPct = .453
That means, using TQI and log5, the Phillies have a 45.3% chance of winning a game against the Rockies.
Some caveats apply. In order to arrive at this result, I had to make a few assumptions. First, I assumed that TQI is the best estimate of a team's actual strength. You might argue that the Rockies are a weaker team now without Jorge De La Rosa, and the Phillies a stronger team with Cliff Lee. But since I am not confident in my abilities to arrive at an exact number making all relevant assumptions, I'll plead ignorance and go with what is a very good estimate.
Now that we have a number that implies the probability of the Phillies beating the Rockies, we can begin to make some inferences about what will happen in a series.
Since I'm a law student, I am subject to the tyranny of the hypothetical. And now, dear friends, you will share in my pain.
Let's say (purely hypothetically!) you have tickets to Game 5 of the NLDS, set to be played on Tuesday. Of course, this game will only be played if it is necessary. What are the odds it will be necessary? First, let's figure out what would need to happen for the game to be played.
There are four sets of outcomes for Games 3 and 4, the games that will determine whether there is a Game 5. As the series is currently tied 1-1, only if one team wins both games this weekend will there be no need for a Game 5. If however, the teams split, a Game 5 will be played. Written out, the possibilities are:
PHL, PHL
PHL, COL
COL, PHL
COL, COL
In the two bolded cases, there would be a Game 5. In the two other cases, there would not. So, the odds are 50%, right? Wrong!
In fact, we already have the information to figure this out, though. Applying the probability already determined to the outcomes above:
PHL, PHL (.453 * .453 = .205)
PHL, COL (.453 * .547 = .248)
COL, PHL (.547 * .453 = .248)
COL, COL (.547 * .547 = .299)
Adding the probabilities of the two middle outcomes, we get a probability of .496. See, I told you it wasn't 50%. It's just ridiculously close to 50%!
So there is a 49.6% chance that our theoretical ticket holder has to find the fastest way to get from New York to Philadelphia next Tuesday.
But what if we want to figure out the odds of a whole series? Here's a little guide you can use once the next round starts.
Let's assume the Angels face off against the Yankees (calm down Red Sox and Twins fans, this is another hypothetical). The Yankees' TQI was .643. The Angels' was .549. Using the above formula, the odds of the Yankees beating the Angels in a single game is .597. So what are the odds the Yankees will prevail in a best-of-seven series?
To figure this out, we can use our old friend, Excel's BINOMDIST function. Here's the syntax
=BINOMDIST(3,7,.403,1)
This is, strictly speaking, calculating the odds that the Yankees do not lose more than three games in the series. To do that, I have told Excel to find the odds of losing up to 3 games (that's the first term) out of 7 (second term), given the probability of the Yankees losing (third term). The last term is telling Excel to count probabilities for the Yankees losing 2, 1, or 0 games as well. When you run this formula, you find that the Yankees have a .704 probability of winning such a series.
You can do this for any series with the aid of only a spreadsheet. Pretty neat, right?
Of course, there are more caveats. To arrive at these numbers, I had to make a few assumptions. First, I ignored home field advantage, which has been shown to be statistically significant. Second, I assumed that the probabilities of winning each game were identical and independent. That is, the outcome of one game does not affect the odds in the next game. Finally, I have ignored the starting pitchers of the teams. This is mostly for simplicity's sake.
I'm glad to report that Clay Davenport's Postseason Odds are back online this year. Rather than using a binomial function, Clay uses a Monte Carlo simulation to calculate odds. Notably, his method DOES include information about the expected starters in each game. Thus, his system is probably more accurate than the one outlined above.
You can also find detailed probabilities at Cool Standings. They will even show you the probabilities of a particular series outcome (say, 3-1 in the ALDS). Of course, using BINOMDIST, you could find this as well!
Discussion Question of the Day
It's been a bad year for forecasting (particularly of the economic variety). How possible do you think it is to predict outcomes of playoff series? Am I being overly sanguine about the possibility? Is it truly a crapshoot?
38 comments
|
0 recs |
Do you like this story?
Comments
When I did my playoff odds, I didn't use a Binomial Distribution to figure out the expected outcomes
The reason being is that we don’t have a perfect measure of a teams true strength, only an estimate. A binomial distribution accounts for random variance from a teams true mean in a sample performance, not for the measurement error of that true mean.
What I did was figured out the standard deviation of the projections I used historically (remember they were just JinAZ’s power rankings regressed to PECOTA preseason projections). Then I used a normal distribution with my standard deviation plugged in. That got me a wider range of possible outcomes than you would get by using a Binomial Distribution.
I believe a Monte Carlo simulation has the same flaws, in that it is too “confident” in the estimate of each teams true talent level.
Interesting
I suppose that would mean the bias was toward better teams winning more often. So, on your view, playoffs are more of a crapshoot than even Clay’s numbers imply?
by Tommy Bennett on Oct 9, 2009 6:02 PM EDT up reply actions
I'd have to see exactly how a Monte Carlo simulation works
If my suspicions are correct, and Clay is being too confident in his estimates each teams true strength, then yes, the playoffs will be more of a crapshoot
Aah, actually
http://www.baseballprospectus.com/article.php?articleid=3490
A second major change was the realization, which dawned slowly and only after several people tried repeatedly to convince me of it, that even these regressed values were estimates, not a hard fact about the team’s future performance. Of course I knew that, but my initial take was that the Monte Carlo simulation itself would supply sufficient variation around the estimate. After more correspondence with people who actually use Monte Carlo simulations as a regular part of their professional career, and more reading on my part, I no longer believe that. There is a real need to recognize, up front, that while I am calling Boston a .550 team, they may in fact be a .650 team, or a .450 team; they may even be a .999 team that has gotten incredibly unlucky, although the odds against that are staggering. The simulation will work better if instead of using the same estimate for Boston’s winning percentage in every run, I let the estimate vary.
How much it should vary is answered, in part, by returning to those regression equations. The standard errors for the estimates were never zero; they varied between .075 and .130, depending on how many games had been played. As a very crude (but simple and easily programmed) solution, I added a random number between -.100 and +.100 (.100 being roughly the standard error) to the team’s winning percentage on every iteration, and it made a big difference. Everything pushed a little farther away from the endpoints, zero and one, and a little closer to .5; the certainties were not nearly so certain anymore. But this was, like I said, a crude solution, and after a little more mathematical effort I’ve replaced it with a system that replicates a Gaussian (normal) distribution around the primary estimate, with a standard deviation of .100; getting the SD to vary with games played is next on the to-do list for this routine, but it isn’t there quite yet.
Emphasis mine. It seems that Clay is doing it right after all, at least in the regular season.
Wow
I literally just finished a spreadsheet using binomdist() to get series odds, updated based on current series leads. I was going to post it tonight once the games were over And then I logged in and saw that you’re already talking about just this sort of thing. :) FWIW, going into tonight’s action, I have series win ’s as 62 for LAA over BOS, 84% for NYY over MIN, 94% for LAD over STL, and 57% for COL over PHI.
Obviously, vivaelpujols’s comments change my plans a bit, though, as the problem he raises makes perfect sense. I’m trying to figure out how to implement it, though, and I’m drawing a blank. Any chance you can walk me through your process? I can come up with an estimate of standard deviation around my estimated true talent levels, but I’m not sure to implement that uncertainty into estimates of how often teams will win games, much less series. Anything you can point me to would be very welcome.
Seems like it should also be possible to get some approximation of this by simply regressing the estimated team talent levels a certain amount toward 50%. No idea how to determine how much to regress, though.
Thanks for any help you can give,
Justin
Seems like it should also be possible to get some approximation of this by simply regressing the estimated team talent levels a certain amount toward 50%. No idea how to determine how much to regress, though.
I’m not so sure about this. What you want is to approximate a normal distribution the mean of which is the estimate of team quality. I haven’t been able to think of a way to do this with BINOMDIST yet, but I think the answer involves doing a sampling of points along a normal curve and then averaging the values.
The tricky part would be that as sample points increased the number of calculations would increase exponentially (because you’d be doing it for each team). I suppose you could just do the BINOMDIST and then do the normal distribution around THAT, but I’m not sure that would achieve the intended result.
Nick may have better ideas than I do, however.
by Tommy Bennett on Oct 9, 2009 10:37 PM EDT up reply actions
When I did mine, I just used the Normal Distribution function in Excel
Using the possible win totals of each team. So I filled in each row 1-162, and calculated the chance that a team reaches each of those totals centered around their projected mean and the observed standard deviation. Mine was simpler than yours would be, because I didn’t consider the relationship between teams, which you have to do.
So what you would do, I think, is this:
1) Find your mean estimated W% for each team
2) Find the standard error of your projections in terms of w% (I don’t know how you would do this, but you found a way I guess)
3) List the range of possible “true talent w%” (I would do intervals of 10 or 20, or else it would take all night).
4) Figure odds that each team is that good using the Normal Distribution centered around your estimated mean of each team and the standard deviation of your projections.
5) Using the Log5 method, consider the odds that one team beats another using all possible combination of “true talent level” for each team
6) Figure out the odds that they both reach those combination in #5 using the steps in #4.
7) Then use the Binomial Distribution function and the method Tommy showed above, calculate each teams chance of winning a series given all possible combination of the probabilities derived in #5.
8) Multiply the results in post 7 by the odds found in post 6. Then sum the results.
Be forewarned, this will take a lot of tedious work, but it should give you excellent odds for any team winning any given series. It might not be much of an improvement to your regression method but it’s more “correct”.
by vivaelpujols on Oct 9, 2009 11:44 PM EDT up reply actions
On second read, these directions don't seem very clear
Let me know if you need any help with any of the steps.
by vivaelpujols on Oct 9, 2009 11:47 PM EDT up reply actions
No, it makes perfect sense
I was just hoping for something less involved.
I think the regression approach I mentioned would work as a shortcut approximation. But I think we’d almost have to run through this whole process to get a good indication of how much to regress. The standard deviation estimate approaches I had would almost certainly be overestimates of the error, too, so it still wouldn’t be “right.”
Anyway, I’m not sure if I’ll do this or not. I’ll have to see how I’m moved tonight. I have a few other projects I want to do, and this was just supposed to be a short-term distraction. :)
-j
If you wanted to get the "right" standard deviations
You could look at historical Third Order Wins data.
by vivaelpujols on Oct 10, 2009 3:50 PM EDT up reply actions
In a short series like this, the pitching rotation is so key to the odds. Here’s how I’ve done it in the past:
- Figure team expected win percentage WITH THAT STARTING PITCHER for every game. (There can be huge swings here, and remember you have to account for each team’s starting pitcher, and then include bullpen strength).
- Once you’ve done that, figure out the likelihood of each scenario that leads to a playoff win. For simplicity, let’s say a team has odds of .600, .550 and .520 to win each game in a three-game set. So you need to do:
.600*.550+.600*(1-.550).520+(1-.600).550*.520 = .585
As the total odds to win the series.
It looks like
WW+WLW+LWW… or more mathematically sound P(W Game 1)P(Win Game 2)+ P(W game 1)P(L Game 2)P(W Game 3)+ P(L Game 1)P(W Game 2)P(W Game 3). the scenarios that lead to a win in a 3 game series.
As Colin said/implied the match-ups have already been accounted for in the .600, .550, and .520 figures.
Colin please step in if I’ve missed the boat somewhere
by stevesommer05 on Oct 10, 2009 9:29 AM EDT up reply actions
I think you’re right that the easiest gains are in including the starting pitchers. I’m tinkering with a way to estimate the winning percentage based on FIP and length of average start.
Do you have a good way?
by Tommy Bennett on Oct 10, 2009 1:59 PM EDT up reply actions
Here's how I do it:
Take an estimation of projected RA. FIP/.92 works okay – I’ve been using something akin to :
(.65*ProjERA + .35*FIP ERA)/.92
Where ProjERA comes from CHONE. xFIP will work a little better than FIP. That’s a junky way of doing it, but Fangraphs RoS ZiPS seemed broken to me and so I improvised a bit. Then figure average length of start and team bullpen RA (you can figure that yourself off the team splits page on Baseball Reference). Then:
(IP_Start*Pit_RA+(1-IP_Start)*PenRA)/9 = DefRA
Then find the team’s runs per game on offense (again, Baseball Refernce will have this). Use Pythag to figure the expected win percentage of that team against a .500 opponent. Do all this again for the opponent. Then use log5 to find expected win percentage for that game.
Yes, this seems eminently reasonable. Thanks.
by Tommy Bennett on Oct 10, 2009 3:47 PM EDT up reply actions
No problem.
And I should note that using this method, the Monte Carlo method becomes somewhat less than useful, in my opinion – since each game has a different win probability, you’re essentially doing a sim over one observation, and as you increase your number of trials you should converge upon the input value pretty rapidly. Maybe I’m wrong, though.
Doing this for my example of whether the Phillies and Rockies will need a fifth game, and assuming that it should be (9-IP_Start) above (this was probably a typo), I get the following results for the Happ/Hammel and Lee/Jimenez matchups:
50.3% chance that a fifth game will be needed (Phillies odds are 46.8% in Game 3 and 56.2% in Game 4).
Sounds about right to me.
by Tommy Bennett on Oct 10, 2009 6:14 PM EDT up reply actions
By the way
I just got lazy and used xFIP. Also there is a rounding error above—should be 50.4%
by Tommy Bennett on Oct 10, 2009 6:18 PM EDT up reply actions
Yeah.
Forgot to mention – add .040 to the win percentage of the home team when you do the log5.
For the truly lazy, just find some betting odds. The Yankees are -178, and the Twins are +168. To convert those to odds:
178/(178+100) = .640
100/(168+100) = .373
.640+.373=1.013
That unsightly .013 is the vig (I think – Vegas Watch is smarter about this stuff than I am). So divide that in half (.0065) and subtract from each to get:
Yankees: .634
Twins: .366
(I rounded the Yanks up and the Twins down so that it sums to a clean 1.)
That’s it – no log5, no accounting for home field, everything else is accounted for. The people in Vegas who do this stuff do pretty good work, at least as good as we’re likely to do messing around with xFIP.
When I first saw the title, I though this was going to be about JC Bradbury's post on Clutch hitting
Yeah
totally random coincidence that we both used the word “fog.”
by Tommy Bennett on Oct 10, 2009 9:31 AM EDT up reply actions
Thinking out loud here, shouldn't be crazy hard to do a Monte Carlo simulation in Excel.
Of course, I really don’t know what the “Monte Carlo” part means – is it anything more than repeated trials to find a probability empirically?
Beyond the Boxscore Not a member? Sign up.
It does if you apply the fix Clay did--albeit in a ad hoc manner
by Tommy Bennett on Oct 10, 2009 10:04 AM EDT up reply actions
Right, just pick a random number for each team in each simulated game...
and therefore a random winning percentage from within the appropriate distribution.
Beyond the Boxscore Not a member? Sign up.
by Sky Kalkman on Oct 10, 2009 10:12 AM EDT up reply actions
The name Monte Carlo simulation comes from the fact that during the 1930s and 1940s, many computer simulations were performed to estimate the probability that the chain reaction needed for the atom bomb would work successfully. The physicists involved in this work were big fans of gambling, so they gave the simulations the code name Monte Carlo.
by Tommy Bennett on Oct 10, 2009 4:21 PM EDT up reply actions
The link below includes a spreadsheet that allows you to run a Monte Carlo simulation of the entire regular season while not forcing you to hold team strength constant — you can put in a standard deviation for each team’s W%, so their assumed true talent changes from iteration to iteration.
I believe Clay uses a similar tool for his Playoff Odds, but after talking to Tango and MGL prior this season it sounds like the standard deviation Clay uses (.075, IIRC) is too high, and should be around .030. I should look into that aspect a bit more before next season, but using some stdev is definitely necessary for an accurate simulation. As mentioned above, using a constant team strength will overrate the chances of good teams and underrate the chances of poor ones.
I remember that link
I’ve been looking around for it, thanks.
by vivaelpujols on Oct 11, 2009 12:38 AM EDT up reply actions
We discussed a lot of that in this FanShot. At least, indirectly. The .075 sounds like the SD of team wins, which is probably too high, yeah. The observed SD of Pythag to actual wins is .024, which is probably more what you need for this.
(For those curious, BTW, the Angels ended up with a difference of .027 between actual and Pythag Win%.)
The standard deviation for projections vs. actual is about .05 if I'm not mistaken
by vivaelpujols on Oct 11, 2009 3:41 AM EDT up reply actions

by 





















