Framing FanGraphs

The sabermetric movement is long past questioning the value of pitch framing. At this point, it is settled doctrine. While pitch blocking and pop time used to be considered the most important defensive skills for catchers, we now know their ability to steal a few extra strike calls carries much more weight.

Last week, FanGraphs updated their metrics to include catcher framing. This is welcome progress, and their valuation of catcher defense should be more in line with Baseball Prospectus, who incorporated framing years ago.

With FanGraphs joining the party, it’s a good time to evaluate some different framing metrics to see how they compare to each other. This includes FanGraphs and Baseball Prospectus as well as StatCorner—a useful site that has its own framing data.

Let’s start with looking at the overall leaders. By averaging all three framing metrics, here are the players who were the most valuable framers in 2018:

Yasmani Grandal takes the top spot, followed closely by Jeff Mathis. This is almost entirely the reason Mathis still has a job. The range of the three metrics can actually be rather large. Looking at Tyler Flowers, Baseball Prospectus liked his pitch receiving 8.6 runs better than StatCorner.

Catchers can giveth and catchers can taketh away. Here are the ones that cost their teams the most runs last year:

According to Defensive Runs Saved, Francisco Lindor saved 14 runs last year. Willson Contreras was even worse than the opposite of Lindor, just from poor pitch framing. Again, we see tremendous variation between the metrics. All three sites have all these players rated as strongly negative, but StatCorner evaluates Salvador Perez’s work 12.7 runs worse than Baseball Prospectus.

Framing is essentially a counting stat. With more opportunities to receive pitches, catchers can gain or lose more framing runs. Therefore, starting catchers will have higher peaks and lower valleys. To get a more accurate representation how each catcher performs, we need to adjust for playing time.

Here are the leaders of the three metrics (averaged once again) adjusted for 1000 pitches received:

Again, this is what keeps Jeff Mathis employed. We also see why the Dodgers felt comfortable letting Yasmani Grandal walk away— Austin Barnes’ framing is top notch.

Here’s the flip side of that leaderboard:

Just as unadjusted metrics skew heavily towards players with more playing time, adjusted metrics can give us extreme examples from players with very little. Andrew Susac only played seven games at catcher last year. He’s a non-roster invite with Baltimore, and he probably won’t make the team. Anyone who can’t crack the Orioles roster truly is not a major leaguer.

Isiah Kiner-Falefa gets something of a pass, given that he spends more time as an infielder. Defense-first backups A.J. Ellis and Michael Perez are more surprising inclusions on the anti-leaderboard.

Regarding those ranges, it’s clear that the metrics vary from each other, sometimes profoundly. Usually, it’s StatCorner that appears to be the outlier. Here are correlation scores for each of the metrics compared to each other:*

• StatCorner and Baseball Prospectus: 0.72

• StatCorner and FanGraphs: 0.68

• Baseball Prospectus and FanGraphs: 0.96

For perspective, easy stats that we all measure the same way would have a correlation score of 1.0. If we compared, say, home run leaders across different sites, the leaderboards would be identical, because it is indisputable that Khris Davis hit 48 and J.D. Martinez 43.

There is no way to tell who’s right or wrong, but Baseball Prospectus and FanGraphs evaluate framing almost as identically as home runs. StatCorner’s metrics differ considerably, though there is still a fairly strong positive correlation. Here’s how all three look together:

StatCorner evaluates framing consistently lower overall. Theirs is the highest framing score for only 18 out of 115 catchers, whereas they were the lowest for 88 of them. They were the median for just six players, and three were tied with zero on all three metrics. (Kyle Farmer, Rocky Gale, and Taylor Davis, all of whom barely played).

The sum of all framing runs across MLB was -322.2 according StatCorner, 33.6 going by Baseball Prospectus, and -0.4 using FanGraphs. This is probably caused by philosophical differences. If you believe framing runs should be compared to MLB average, the sum of everything should be zero. If you believe that poor framing can cost more runs than good framing can prevent, the number does not have to be zero. However, given that StatCorner’s metric is called Runs Above Average, they should probably adjust upward by about 322 runs.

Overall defense is arguably the most difficult aspect of player performance to evaluate. UZR says Anthony Rendón made a positive contribution of 5.9, but DRS gave him a -6. Baseball Prospectus’ FRAA rates him at -5.7, while Baseball-Reference credits him with -0.4 dWAR. That’s four different ways to measure defensive value, and we still have no idea whether Rendón was good or bad!

The nuance of catching is vastly different than any other position. From the dawn of baseball until about a dozen years ago, we completely missed the mark. Framing is one of the most important defensive skills to evaluate correctly. It’s encouraging to see strong correlations from three different metrics, especially Baseball Prospectus and FanGraphs. Now that both major stat sites evaluate framing so similarly, we can be confident that public framing data is some of the most accurately measured defensive data available.

---

*For correlations, data for all 115 catchers who caught more than one inning in 2018 was used. Sorry, Joe Mauer and Jarrod Saltalamacchia.

---

Daniel R. Epstein is an elementary special education teacher and president of the Somerset County Education Association. In addition to BtBS, he writes at www.OffTheBenchBaseball.com. Tweets @depstein1983