clock menu more-arrow no yes mobile

Filed under:

Are SABR's Changes to the Gold Glove Bad for Baseball?

SABR is teaming up with Rawlings to index existing defensive metrics for use by the Gold Glove and Platinum Glove Awards. But is this really progress?

By Matt Hunter
Nick Laham

At the SABR Analytics Conference in Phoenix, SABR announced that it would team up with Rawlings in order to: 1) create a new fielding metric called SABR Defensive Index (SDI) and 2) use SDI as one contributing factor to determine the Gold Glove Award.

Last week, SABR announced more details about the project. Here's the significant excerpt from the press release:

Gennaro said a high priority for the new Fielding Research Committee would be to establish a transparent index that integrates respected fielding metrics from multiple sources.

"The baseball community's knowledge of defense, and the data we use to evaluate it, is improving every day," Gennaro said. "In keeping with SABR's mission, and the goal of the Analytics Conference, we hope the SDI will be an ongoing effort to integrate the best research and the best methods available to assess defensive performance."

The key points we can draw from this are the following:

  • SDI will integrate various advanced fielding metrics (presumably metrics like UZR, DRS, and FRAA)
  • SDI is an index, meaning it will normalize the metrics so that they are on the same scale and can be compared and integrated as such.
  • SDI will be transparent -- that is, hopefully, the formula and calculations will be open to the public.
There has been quite the uproar among some in the sabermetric community about the integration and indexing aspects of SDI. I have no idea whether this uproar is justified or not, but I suspect that it is. You can't just combine a bunch of metrics, each of which are calculated differently, together and call it progress. That's not how math, or rational thinking, works.

However, progress for sabermetrics and progress for the Gold Gloves -- and baseball as a whole -- are very different. The Gold Gloves, as we all are well aware, don't give us a very good indication of a player's fielding ability. They are biased towards veteran players, and do a poor job at rewarding range and other aspects of defense outside of fielding percentage.

So any semblance of objectivity, stats, and analytics would necessarily improve it, right? Well, not necessarily. First of all, there is still the danger of SDI being so arbitrary and flawed that it adds nothing, or even negatively affects, the Gold Glove's correlation with defensive performance.

I don't see this as terribly likely, however, as defensive metrics are generally considered to be somewhat accurate, albeit noisy, descriptors of defense. Using them as a factor in the Gold Glove is unlikely to worsen the award.

But there are other reasons to be wary of SDI, even if it would improve the accuracy of the Gold Glove.

SDI will incorrectly be viewed as sabermetric progress

The idea behind this argument is that people will believe that SDI is some big sabermetric discovery, when it is really just a combination of existing, flawed, defensive metrics. This complaint certainly has merit -- we saw first-hand this past season the extent to which fans can misuse and misunderstand WAR. Why would SDI be any different? Why would we expect people to understand that it is no better, and is in fact worse, than what we already have, and that it in no way is the end-all of defensive evaluation?

On the other hand, maybe we're not giving fans enough credit here. On Twitter, BP's Colin Wyers rightly accused me of being condescending for saying that SDI would be progress for mainstream understanding of defense. But though he is completely right, that same line of thinking also applies when talking about a mainstream fan's understanding of SDI. If, and this is a big if, SABR is completely transparent about SDI, we should trust that fans will understand the limits of the metric.

Settling for a sub-optimal defensive metric will make it difficult to improve the Gold Glove going forward

If we settle for SDI as a less than ideal, but still positive, factor in the GG award, it may be difficult to improve it further once we have a better grasp of defensive metrics. Why not wait and try to figure out a better way to measure defense before including advanced metrics in the Gold Glove?

This argument would make sense, if not for the wording in the press release from SABR. Vince Gennaro said that SDI will be "an ongoing effort to integrate the best research and the best methods available," an indication to me that it will not be a set-in-stone formula, but simple a placeholder to combine the best defensive metric into one easy-to-read number. If someone makes a groundbreaking discovery about how to measure defense, SABR and Rawlings won't have to wait for another Gold Glove revamp in order to include it -- they can simply add said metric to SDI.

This leads to my third, and, I believe, most convincing objection:

There is no need to combine all the metrics into one number

Sure, UZR and DRS and FRAA are all flawed in one way or another, and sure, using them in the Gold Glove is not the ideal answer. Nevertheless, there is clearly no analytical benefit to combining them all together other than for the sake of simplicity. The Gold Glove can still include these metrics without creating The One Number that will say everything about a player's defense.

If, instead, SABR simply provided a guide that included the metrics -- as they said they would do -- but did not create an all-encompassing metric, there would be less risk of misunderstanding and more room for progress. By separating these metrics, the implication is that none are the perfect answer, and that they each give us a different answer. Voters and fans can then use the metrics as they see fit, but will have a clear understanding that no individual number says everything about a player's defense.

I fully understand the concerns coming from many members of the sabermetric community regarding the SABR Defensive Index, especially those who were already skeptical of defensive metrics to begin with. Nevertheless, though SDI is certainly not progress within the sabermetric community, it would likely improve the accuracy of the Gold Glove Award.

The issue with SDI lies in its potential to confuse fans and voters, damage the credibility of sabermetrics, and slow real progress in measuring defense. However, I believe that these concerns mainly lie in the fact that SDI combines the defensive metrics available, rather than leaving each one as publicly available and independent of one another.

If we can all agree that both the Gold Glove methodology and advanced defensive metrics need to be improved, we, as a sabermetric community, should worry less about SDI and more about educating the public about the pros and cons of defensive metrics available and working towards more accurate, understandable, and transparent methods of evaluating defense.

Btbs-twitter-insert_medium

More From Beyond the Box Score