Which hot and cold starts are for real?

May 1 is an important date for baseball fans. As the book closes on April, the Small Sample Size Brigade starts to recede, the weather in the Northeast becomes less hostile, and certain fan bases start panicking about their underperforming team. As a Red Sox fan, this is a tradition that dates back to my childhood, and one that I assume I will pass on to my children, like my father and his father before him.

Of course, the more rational baseball fan will realize that there are still plenty of games remaining for teams to regress to their true talent level, and Scott Lindholm's recent series showed that even the hottest starts don't necessarily guarantee a playoff berth. But how to balance the results from previous seasons with the handful of games already played?

When he was still writing for Baseball Prospectus, Rany Jazayerli developed a simple system to make rest-of-season predictions. The system is a weighted average of a team's results over the past few seasons and their results over the first few games. As the season progresses, more weight is assigned to this season's games, and less to the assumed "true talent level" derived from previous seasons.

The system consists of two equations. First, a team's projected winning percentage P is given as

P = .1557 + (.4517 * X₁) + (.1401 * X₂) + (.0968 * X₃),

where X₁ is a team's winning percentage from last season, X₂ is a team's winning percentage from two years ago, and X₃ is a team's winning percentage from three years ago. Jazayerli determined that previous seasons had no predictive power for the current season.

If we think of the constant term as the weight given to regression towards .500, our projection is made up of the following components:

This estimates a team's record using previous seasons alone. To incorporate this season's results into a final prediction Y, we use the following formula.

Y = P + ((S - P) * (.0423 + (.0095 * G)))

Here, our team has a winning percentage of S in its first G games.

Now that we've established our method, let's see if it's still accurate by using some out-of-sample data. I compiled the standings from the past three seasons on May 1, June 1, and July 1, used Jazayerli's formula to estimate each team's final winning percentage, and compared the estimate to their final results. First, the mean error, in terms of wins:

This means that, over the past three years, this method overestimates a team's performance by an average of three-thousands of a win, though as we will see, there are plenty of missed calls. Since the means are around 0, this suggests there is no obvious bias: Jazayerli's method is neither consistently overestimating nor underestimating team performance over the rest of the season.

These scatter plots show the distribution of errors over time. We can see that, as the season progresses, the expected winning percentages begin to fall more in line with the final results. This method explains a respectable 42 percent of the variation in final records as of May 1, then increases to 55 percent as of June 1 and 69 percent as of July 1.

This method will never be perfect, of course. The following table shows the worst predictions at the end of the last three Aprils. We can see that, even though the system has no overall bias, it is still capable of producing large errors for individual teams. The teams on this list provide good insight into some of the problems with a regression-only approach: this system does not account for personnel moves, it does not try to predict breakout performances, and -- being regression-heavy -- it will tend to force very good and very bad teams back towards .500 regardless of actual talent.

To better appreciate these pitfalls, consider the 2012 Red Sox, whose final results were 16 games worse than what was predicted on May 1. The chart below tracks their expected and actual winning percentage by date. We see that, even as the team went 12-19 in their first 31 games, Jazayerli's method still expected the team to play .500 ball based on the strength of their past few seasons. And they actually did, hovering right around the break-even point until early August. Of course, once August hit, half the team got injured, the other half got traded to the Dodgers, and the front office started trying to get Bobby Valentine to say his name backwards in the hopes of sending him back to his home planet.

Next, let's look at last season's Pirates, a much happier example. On May 1, 2013, the model predicted just over 81 wins for the Pirates, 13 under their final total. Here, the graph shows a season-long skepticism that melted as the Pirates continued to win. When the team was just under .600 at the end of April, the model projected the Pirates to finish at .500. But as the team continued to win, the model gradually warmed to the team, until by the All-Star Break, it too estimated the Pirates as a .600 ballclub*.

* - We can also see from both of these graphs that, around mid-July, the model falls apart, as too much weight is placed on the difference between the projected and actual record. But if you're still hoping your team is a regression candidate in August, it's probably time to start thinking about next year.

Now that we trust the model, and understand its flaws, let's see what it thinks of this year's hot and cold starts. Here, the Delta column represents the difference between a team's current winning percentage (as of this morning) and its predicted winning percentage. Note that the projected record is rounded to the nearest game.

Not surprisingly, the model favors those teams off to incredibly slow and incredibly fast starts to regress back towards .500, although even after the regression, the model still expects the Braves to win around 95 games and the Diamondbacks to lose over 90.

. . .

All statistics courtesy of Baseball-Reference.

Bryan Cole is a featured writer at Beyond the Box Score and is just now thawing out from watching a game at Wrigley Field last week. You can follow him on Twitter at @Doctor_Bryan.