Or perhaps, framing, the debate.

The defensive influence of the catcher has long been debated, with sabermetric circles generally valuing it less than baseball insiders. It's possible to identify seven separate components as places where a catcher can contribute defensively: stolen bases, blocking pitches, blocking the plate, fielding bunts (and psuedo-bunts), game calling, pitch framing and pitcher preparation. Many people have made attempts at quantify different pieces of the overall catcher contribution. Ability to prevent stolen bases and fielding ability are included in a number of measures, such as UZR and Win Shares. Keith Woolner from Baseball Prospectus has examined the topic of game calling a few times and has placed an upper limit of the effect at .60 runs of ERA. I've attempted to measure how well catchers block pitches in the dirt. Blocking the plate is very difficult to measure since it occurs relatively rarely and success can be quite subjective (the runner might be out, but the catcher did nothing to impede him from reaching the plate). Determing a catcher's skill at blocking the plate is probably best left to visual analysis. Pitcher preparation, which I'm using to mean any interaction the catcher has with his staff, from going over game plans, to calming the pitcher during visits to the mound, is another area that would be nearly impossible to objectively measure, but almost certainly has some effect on the game. One area that hasn't been explicitly covered (I suspect many would roll it into game calling) is pitch framing.

I've attempted a study to measure the effects of framing pitches. I say attempted because, to be honest, the results seem wrong to me, but I can't figure out where the issue lies. I present the rest of this article in the hopes that someone can take the information and make sense of it. Some may say I shouldn't publish this until I'm more comfortable with the outcome, but I'm a big fan of collaboration and knowledge sharing, so I'm going ahead even though I don't know the answer. Ok, enough honesty, let's move on.

Back around Thanksgiving, Jonathan Hale broke down the strike zones of various umpires over at The Hardball Times. He used the PITCHf/x data and the strike zones figured by John Walsh to determine how many strikes above or below average an umpire called in the course of a game (for his purposes, 150 called pitches). The results ranged from -5.2 strikes for Gerry Davis to +4.81 strikes for Jeff Nelson, where negative numbers implies a smaller strike zone.

A similar methodology can be used to examine whether catchers have any effect on whether the pitch is called a ball or strike. Using Walsh's strike zones, empirically defined as the areas where at least 50% of pitches are called strikes, I determined for each catcher how many balls should have been strikes and vice versa. This allowed me to calculate an average rate and then figure out how many strikes above or belowe average each catcher was. The results for those catchers with more than 2000 called pitches (roughly 120 innings using Hale's 150 called pitches per game number) can be seen below. The run value is calculated using the run value for changing a ball to a strike - .161 runs, which I explain here.

I'll be the first to admit this is a much larger effect than I expected to see. In fact it's so large that I have to think there's something wrong in the analysis. Over the course of 120 games (a reasonable estimate for the number of games caught in a season by a starting catcher), the difference between Gregg Zaun and Gerald Laird is over 250 runs or 25 wins. The per game numbers, though, are similar to what Hale found for umpires. So either this style of analysis is lacking, or my run values are off.

But what about the effect of the umpire on this? After all, my study is based on Hale's work with the umpires. I did a weak correction for the umpires by using his numbers and determining what the expected strike delta would be for each catcher / umpire pair and subtracting that effect from the catcher's individual numbers. Since he only published umpires with a sample size greater than 20 games, I assumed the other umpires had 0 effect. This obviously gets into more drastic sample size issues than does the initial study, but I thought it would be interesting to look at.

Adding the influence of the umpire seems to dampen the effect somewhat, which is what we'd expect. Still, the spread between best and worst is huge. We're no better off than we were before.

What to make of all this? There are obvious quality and sample size concerns from the PITCHf/x data, which calls some of this into question. Beyond that, the results just seem too outlandish to be correct. Plus, there was only a weak correlation between these values and the team's runs allowed per game (r=-.30 based on those catchers who caught for the same team all year). I find it hard to believe pitch framing can have this big an impact and not be more noticeable. I have the same concern about the results of Hale's analysis of umpires - where the impact is nearly as big. Two other possibilities are that my run value number is wrong (likely, but I think it's in the ballpark) or that there's some underlying issue that affects both of our studies. Finally, maybe we've been wrong the whole time and catcher defense is really that important.