Framing the Debate

Or perhaps, framing, the debate.

The defensive influence of the catcher has long been debated, with sabermetric circles generally valuing it less than baseball insiders. It's possible to identify seven separate components as places where a catcher can contribute defensively: stolen bases, blocking pitches, blocking the plate, fielding bunts (and psuedo-bunts), game calling, pitch framing and pitcher preparation. Many people have made attempts at quantify different pieces of the overall catcher contribution. Ability to prevent stolen bases and fielding ability are included in a number of measures, such as UZR and Win Shares. Keith Woolner from Baseball Prospectus has examined the topic of game calling a few times and has placed an upper limit of the effect at .60 runs of ERA. I've attempted to measure how well catchers block pitches in the dirt. Blocking the plate is very difficult to measure since it occurs relatively rarely and success can be quite subjective (the runner might be out, but the catcher did nothing to impede him from reaching the plate). Determing a catcher's skill at blocking the plate is probably best left to visual analysis. Pitcher preparation, which I'm using to mean any interaction the catcher has with his staff, from going over game plans, to calming the pitcher during visits to the mound, is another area that would be nearly impossible to objectively measure, but almost certainly has some effect on the game. One area that hasn't been explicitly covered (I suspect many would roll it into game calling) is pitch framing.

I've attempted a study to measure the effects of framing pitches. I say attempted because, to be honest, the results seem wrong to me, but I can't figure out where the issue lies. I present the rest of this article in the hopes that someone can take the information and make sense of it. Some may say I shouldn't publish this until I'm more comfortable with the outcome, but I'm a big fan of collaboration and knowledge sharing, so I'm going ahead even though I don't know the answer. Ok, enough honesty, let's move on.

Back around Thanksgiving, Jonathan Hale broke down the strike zones of various umpires over at The Hardball Times. He used the PITCHf/x data and the strike zones figured by John Walsh to determine how many strikes above or below average an umpire called in the course of a game (for his purposes, 150 called pitches). The results ranged from -5.2 strikes for Gerry Davis to +4.81 strikes for Jeff Nelson, where negative numbers implies a smaller strike zone.

A similar methodology can be used to examine whether catchers have any effect on whether the pitch is called a ball or strike. Using Walsh's strike zones, empirically defined as the areas where at least 50% of pitches are called strikes, I determined for each catcher how many balls should have been strikes and vice versa. This allowed me to calculate an average rate and then figure out how many strikes above or belowe average each catcher was. The results for those catchers with more than 2000 called pitches (roughly 120 innings using Hale's 150 called pitches per game number) can be seen below. The run value is calculated using the run value for changing a ball to a strike - .161 runs, which I explain here.

Catcher Called Pitches SAA SAA / 150 Pitches Runs / 150 Pitches
Gregg Zaun 4518 159.70 5.30 0.85
Jeff Mathis 3430 93.43 4.09 0.66
Yadier Molina 3246 84.49 3.90 0.63
Russell Martin 7061 183.16 3.89 0.63
Yorvit Torrealba 2867 58.03 3.04 0.49
John Buck 3443 64.93 2.83 0.46
Brian McCann 5762 105.98 2.76 0.44
Javier Valentin 2013 36.79 2.74 0.44
Jose Molina 2313 41.28 2.68 0.43
Brian Schneider 2119 31.72 2.25 0.36
Ivan Rodriguez 3881 55.13 2.13 0.34
Brad Ausmus 2576 35.19 2.05 0.33
Johnny Estrada 3072 36.16 1.77 0.28
Carlos Ruiz 2471 11.22 0.68 0.11
Benji Molina 3909 14.06 0.54 0.09
Jorge Posada 2616 3.66 0.22 0.03
Paul LoDuca 2569 -0.54 -0.03 -0.01
A.J. Pierzynski 5932 -3.54 -0.09 -0.01
Victor Martinez 3233 -2.01 -0.09 -0.02
Josh Bard 5701 -15.68 -0.41 -0.07
Kurt Suzuki 4269 -15.75 -0.55 -0.09
Jason Varitek 3543 -14.90 -0.63 -0.10
Jason Kendall 5770 -25.32 -0.66 -0.11
Michael Barrett 2499 -15.86 -0.95 -0.15
Chris Snyder 3342 -23.19 -1.04 -0.17
Ronnie Paulino 2343 -16.87 -1.08 -0.17
Rob Bowen 2057 -26.90 -1.96 -0.32
Mike Napoli 3106 -45.14 -2.18 -0.35
Jamie Burke 2270 -35.07 -2.32 -0.37
Ramon Hernandez 2361 -39.56 -2.51 -0.40
Jarrod Saltalamacchia 2467 -41.63 -2.53 -0.41
Joe Mauer 2434 -44.36 -2.73 -0.44
Dioner Navarro 2436 -64.44 -3.97 -0.64
Kenji Johjima 7120 -201.11 -4.24 -0.68
Gerald Laird 6129 -317.09 -7.76 -1.25

I'll be the first to admit this is a much larger effect than I expected to see. In fact it's so large that I have to think there's something wrong in the analysis. Over the course of 120 games (a reasonable estimate for the number of games caught in a season by a starting catcher), the difference between Gregg Zaun and Gerald Laird is over 250 runs or 25 wins. The per game numbers, though, are similar to what Hale found for umpires. So either this style of analysis is lacking, or my run values are off.

But what about the effect of the umpire on this? After all, my study is based on Hale's work with the umpires. I did a weak correction for the umpires by using his numbers and determining what the expected strike delta would be for each catcher / umpire pair and subtracting that effect from the catcher's individual numbers. Since he only published umpires with a sample size greater than 20 games, I assumed the other umpires had 0 effect. This obviously gets into more drastic sample size issues than does the initial study, but I thought it would be interesting to look at.

Catcher Opportunities SAA SAA / 150 Pitches Runs / 150 Pitches
Gregg Zaun 4518 167.81 5.57 0.90
Yadier Molina 3246 83.25 3.85 0.62
Russell Martin 7061 167.61 3.56 0.57
Jeff Mathis 3430 77.43 3.39 0.55
Jose Valentin 2013 42.55 3.17 0.51
Jose Molina 2313 46.95 3.04 0.49
John Buck 3443 63.28 2.76 0.44
Brian McCann 5762 98.41 2.56 0.41
Brad Ausmus 2576 41.80 2.43 0.39
Yorvit Torrealba 2867 44.57 2.33 0.38
Ivan Rodriguez 3881 55.67 2.15 0.35
Brian Schneider 2119 29.76 2.11 0.34
Johnny Estrada 3072 32.23 1.57 0.25
Jorge Posada 2616 21.64 1.24 0.20
Benji Molina 3909 22.23 0.85 0.14
Paul LoDuca 2569 2.11 0.12 0.02
Carlos Ruiz 2471 1.56 0.09 0.02
A.J. Pierzynski 5932 2.11 0.05 0.01
Victor Martinez 3233 -4.29 -0.20 -0.03
Kurt Suzuki 4269 -6.32 -0.22 -0.04
Jason Varitek 3543 -5.81 -0.25 -0.04
Jason Kendall 5770 -21.46 -0.56 -0.09
Josh Bard 5701 -31.55 -0.83 -0.13
Ronny Paulino 2343 -15.23 -0.98 -0.16
Michael Barrett 2499 -20.02 -1.20 -0.19
Chris Snyder 3342 -34.53 -1.55 -0.25
Mike Napoli 3106 -42.51 -2.05 -0.33
Jamie Burke 2270 -33.45 -2.21 -0.36
Rob Bowen 2057 -30.50 -2.22 -0.36
Ramon Hernandez 2361 -35.80 -2.27 -0.37
Jarrod Saltalamacchia 2467 -48.45 -2.95 -0.47
Joe Mauer 2434 -48.64 -3.00 -0.48
Dioner Navarro 2436 -63.65 -3.92 -0.63
Kenji Johjima 7120 -189.77 -4.00 -0.64
Gerald Laird 6129 -314.00 -7.68 -1.24

Adding the influence of the umpire seems to dampen the effect somewhat, which is what we'd expect. Still, the spread between best and worst is huge. We're no better off than we were before.

What to make of all this? There are obvious quality and sample size concerns from the PITCHf/x data, which calls some of this into question. Beyond that, the results just seem too outlandish to be correct. Plus, there was only a weak correlation between these values and the team's runs allowed per game (r=-.30 based on those catchers who caught for the same team all year). I find it hard to believe pitch framing can have this big an impact and not be more noticeable. I have the same concern about the results of Hale's analysis of umpires - where the impact is nearly as big. Two other possibilities are that my run value number is wrong (likely, but I think it's in the ballpark) or that there's some underlying issue that affects both of our studies. Finally, maybe we've been wrong the whole time and catcher defense is really that important.

Log In Sign Up

Log In Sign Up

Forgot password?

We'll email you a reset link.

If you signed up using a 3rd party account like Facebook or Twitter, please login with it instead.

Forgot password?

Try another email?

Almost done,

By becoming a registered user, you are also agreeing to our Terms and confirming that you have read our Privacy Policy.

Join Beyond the Box Score

You must be a member of Beyond the Box Score to participate.

We have our own Community Guidelines at Beyond the Box Score. You should read them.

Join Beyond the Box Score

You must be a member of Beyond the Box Score to participate.

We have our own Community Guidelines at Beyond the Box Score. You should read them.




Choose an available username to complete sign up.

In order to provide our users with a better overall experience, we ask for more information from Facebook when using it to login so that we can learn more about our audience and provide you with the best possible experience. We do not store specific user data and the sharing of it is not required to login with Facebook.