Last week, I documented the different methodologies of the big projection systems, and unsurprisingly, different methodologies lead to different results. Most of the time, there's still general agreement on a player, with the disagreements coming around the edges. However sometimes, in the extreme cases, the difference in overall value gets up to multiple wins. The same is generally true of teams, in that the various projection systems might not exactly agree, but probably have a shared sense of what's possible for a team. Then there's this:
The former, from FanGraphs' projected standings; the latter, from Baseball Prospectus'. I probably don't need to point out the gap, but here I go anyway: the Rays are projected as a .500 team by FanGraphs (which is good for fourth in the competitive AL East), and as a .562 team by Baseball Prospectus (which is good for first). In terms of wins and losses, it's a ten-game swing.
That struck me as a massive gap, and indeed it is; the next-largest difference in win percentage between FG and BP is .038, for the Brewers, which translates to a mere six games. More than that, though, the Rays projections disagree about the path their season will follow. The Brewers' gap isn't small, but it's not particularly meaningful: they're projected for fourth in their division either way. For all non-Rays teams, FanGraphs and Baseball Prospectus don't disagree on their projected place in the division by more than one spot. The combination of a large disagreement and a tight division make this a huge swing from one projection to the other.
When faced with something like this, you can do a few things. You can just pick one – optimism if you're a Rays fan, pessimism if you're a Red Sox or Blue Jays fan – and not worry about it. You can average the two, which, while blunt, is not the worst tactic. Or you can try to dig into the details of the projections, learn something about the Rays from the disagreement, and draw a reasoned conclusion. The first two methods don't make for a very substantial article.
Both FanGraphs and Baseball Prospectus project each team's runs scored and runs allowed, which can give a broad overview of where the difference lies. FanGraphs projects the Rays to be 2 percent below average in runs scored and the same amount below average in runs allowed (hence the .500 projection). Baseball Prospectus on the other hand thinks Tampa will be 4 percent above average in runs scored and 9 percent better than league average in runs allowed: a six-point and seven-point swing, respectively. It doesn't really help narrow down where the difference is coming from, however, so the next step is to look at the projections of individual players.
When we look at the rotation, there's a bit of a surprise. FanGraphs projects every single Rays starter for more WAR than Baseball Prospectus: Chris Archer (4.1 fWAR vs. 2.9 WARP), Jake Odorizzi (2.5 vs. 1.9), Drew Smyly (2.4 vs. 1.6), Erasmo Ramirez (1.3 vs. 1.0), and Matt Moore (1.0 vs. 0.6). This is unintuitive, given that Baseball Prospectus projects the Rays to prevent more runs than FanGraphs does, but the answer lies in the way the two sites calculate and distribute WAR.
FanGraphs' pitcher fWAR is based on Fielding-Independent Pitching, which treats outcomes on balls in play as essentially random, and just grades pitchers on their strikeouts, walks, and home runs. Baseball Prospectus' WARP however, is based on Deserved Run Average, a stat based on a complex mixed model that assigns credit to a pitcher for every event, adjusted to a wide variety of other factors, like the framing ability of the catcher, who the opposing batter is, the temperature, and the defense behind the pitcher.
Among other things, this allows BP to give credit to catchers for framing, credit which is taken away from the pitcher. Indeed, that's where the vast majority of this gap comes from; FanGraphs' projects Curt Casali, Rene Rivera, and Hank Conger to be slightly below average on defense, but that's only considering blocking and throwing. Baseball Prospectus projects Conger to be excellent and Rivera to be good at framing, boosting their respective value considerably, and cutting some value out from the pitching staff in return.
On traditional defense, both Baseball Prospectus and FanGraphs are very high on the Rays. The former is slightly more aggressive, thinking the Rays will be league-leaders by a substantial margin, while the latter puts them in a crowded second tier. When all run-prevention is taken into account, however, Baseball Prospectus ends up even more optimistic, mostly because of framing.
On offense, there's no single player that Baseball Prospectus loves that FanGraphs doesn't, but there are a series of positive projections that add up to a big difference in prediction. The former thinks Evan Longoria will age slightly more gracefully than the latter, and produce a better OBP, SLG, and defense; and that Steven Souza will continue to improve rather than performing around his career averages. One of the biggest gaps comes in Corey Dickerson's projection, who FanGraphs thinks will be a useful platoon piece, while Baseball Prospectus thinks that he'll get more playing time and be more productive when he does. Almost every single Rays player gets a more optimistic treatment by PECOTA, with the exception of James Loney, who neither system sees as good but FanGraphs sees as passable.
In other words, there's no one event to peg this 10-game gap on. The biggest culprit appears to be framing, so which projection you trust more might depend on how you feel about that metric. If you think it's fading in importance as all teams pick up on it, or as umpires begin to pay more attention to it, you might prefer FanGraphs' forecast; but if you think framing remains an undervalued way for low-budget teams like the Rays to pick up value, you'll likely prefer Baseball Prospectus'. Otherwise, it's just a general sense of sanguinity across the board.
Where does that leave a bemused fan, seeking a single number to use? Probably back at averaging the two, to be honest. When given two pieces of conflicting information, with little reason to prefer one over the other, it makes the most sense to incorporate them both. Personally, I am a believer in Baseball Prospectus' framing data, so I might weight their forecast slightly more heavily than FanGraphs. In the end though, you'll drive yourself crazy trying to extract clear conclusions from projections. They're uncertain by their very nature, and while a ten-game difference before the season is large, they're frequently off at the end of the year by far more than ten games. It's better to focus on what information they can share – the Rays care about framing, a lot! Their season might depend heavily on how well they perform on defense – than what they can only offer an educated guess at.
. . .
Henry Druschel is a Contributing Editor at Beyond the Box Score. You can follow him on Twitter at @henrydruschel.
Loading comments...