Judging the experts' World Series predictions

The start of the World Series last week brought out a plethora of expert predictions. Writers, broadcasters, bloggers, and simulators put forth their best guesses as to who would win, and how many games the series would last.

The large number of predictions brought with it speculation about the distribution of the predictions. On Twitter, Baseball Prospectus editor-in-chief Sam Miller wondered if media members' affiliations affected their forecasts.

Hypothesis: Most team broadcasters pick [Team] in 7 Games; most writers pick [Team] in 6; most on-air pundits/personalities pick [Team] in 5

— Sam Miller (@SamMillerBP) October 21, 2014

I enjoy aggregating predictions; I've previously collected expert picks for the Super Bowl and the NCAA basketball tournament and ran a calibration test on Nate Silver's election forecasts. So I found 51 expert predictions before the start of the series, all but two of which included the number of games the expert expected the series to last.

And a handful of experts hit the nail right on the head: six experts* (Gordon Edes, David Kull, Andrew Marchand, David Schoenfield, Matt Snyder, and Tom Verducci) and one Out of the Park simulation picked the Giants to win in seven games.

* - I'm not counting Craig Calcaterra, who's on the record predicting both the Royals in seven and the Giants in seven.

Now admittedly, 50 is not a representative sample of media personalities, and there isn't enough diversity in the sample to investigate Miller's hypothesis. But it's still large enough to find patterns. The graph below gives the percentage of experts who picked each combination of victor and series length (in blue).

For comparison's sake, I included theoretical probabilities based on the log5 formula (in red). I used the Pythagenpat formula to estimate each team's talent level*, and a home-field boost of three percent (based on the .530 winning percentage of all teams this year in home games). Since there are only a limited number of possible series outcomes, I could compute these probabilities analytically rather than estimate them through a simulation.

* - By this method, the Giants have a 55 percent win probability when facing the Royals at AT&T Park, and a 49 percent WP at Kauffman Stadium.

Comparing these two, we see some large discrepancies. First, we note the lack of sweeps: The theoretical approach suggests that 12% of the time, the Royals or Giants would have won in a sweep. Yet only one writer -- ESPN's Jim Caple -- predicted a sweep when asked.

The other feature worth remarking on is the two clusters: those who favor the Giants tended to pick them to win in five, whereas those who backed the Royals were more likely to pick them to win in six.

I've also included the historical probabilities in black. For example, the favorites won 15 of the 100 seven-game series played since 1969 in seven games, while the underdogs have won five series by sweep.

There are a number of discrepancies between the historical and theoretical percentages, but that doesn't mean the odds ratio is worthless in the playoffs. This was just a raw count, with favorites assigned by Pythagenpat winning percentage. A more thorough investigation would take home-field advantage and relative strength into account.

But even if the historical and theoretical percentages matched up perfectly, we still shouldn't expect the experts to agree perfectly. For one thing, each individual media member will pick the result they think is most likely, leading to an unexpected peak around the group consensus. We can see an example of this in the Super Bowl predictions I linked to above: the overwhelming majority of analysts predicted a 49ers win, even though most predicted a close game. Although a 60-40 split might have better matched the general consensus, each media member was incentivized to pick the slight favorite.

Besides, predicting the length of a series is even more of a fool's errand than predicting the result of an individual game. In that sense, it might be more logical to think of the number of games as a confidence score: picking a team to win in a sweep, for example, implies much more confidence in your prediction than picking the same team to win in seven games.

. . .

Predictions courtesy Baseball Prospectus, Bleacher Report, CBS Sports, ESPN, FanGraphs, Grantland, NBC Sports, Out of the Park, Sports Illustrated, and The Sporting News.

Bryan Cole is a featured writer for Beyond the Box Score, and is already staring out the window and waiting for spring. You can follow him on Twitter at @Doctor_Bryan.