clock menu more-arrow no yes mobile

Filed under:

How well do umpires call balls and strikes?

PITCHf/x data isn't just for pitchers--it allows umpires to be evaluated as well. You might be surprised at the numbers.

Umpire Vic Carapazza makes a strike call
Umpire Vic Carapazza makes a strike call
USA TODAY Sports

I wrote a post last week on how often pitchers gets calls on pitches outside the strike zone. It had far too many moving parts, but the idea still intrigued me. I realized there was a simpler way to convey this information, which was to investigate two questions:

1. How often was a pitch inside the strike zone called a ball?

2. How often was a pitch outside the strike zone called a strike?

I'll begin with this chart showing the location of all pitches in 2013 according to data at baseballsavant.com--white is the strike zone, yellow outside the strike zone:

Umpire_2_medium

It helps to understand how many pitches are actually called by umpires as opposed to put in play, fouled off or swung at for a strike. This table shows the data from 2008-2013:

Strike_zone_2_medium

Just over half of pitches required a call by the umpire. Since Daren Willman at baseballsavant.com allows for PITCHf/x searches by umpire, we can check and see how often they made the correct call. This table breaks down all called balls and strikes from 2008 to 2013:

Strike_zone_1_medium

Just to be clear, 13.2% of pitches inside the strike zone were called balls and 86.8% were correctly called strikes. Likewise, similar numbers of pitches were called correctly and incorrectly. It's beyond my ability to determine how close any of these pitches were to the strike zone, but common sense and watching games on TV suggests the "bad" calls were on pitches right on the edge of the strike zone.

I was also curious if there trends in calling strikes above or below the strike zone. This chart shows the data:

Strike_zone_3_medium

With only 6 years of data I'm not going to make grand pronouncements. I was more interested in seeing if there was a dramatic difference between high strikes and low ones, and the data doesn't appear to support that conclusion.

How did individual umpires perform? This chart shows all umpires with 15% or more "bad" calls:

Strike_zone_4Click on image to enlarge

The first group of columns shows the percent of total pitches umpires make calls on--strikingly similar. The next three groups show pitches in the strike zone that were called balls and high and low pitches that were called strikes.

Who were the best? This chart lists them, leaving out any umpire seeing fewer than 10,000 pitches:

Strike_zone_5Click on image to enlarge

I don't like using terms like "bad" or "wrong" when reviewing data like this because calling balls and strikes is difficult. It appears umpires make the wrong call around 15% of the time, which is just too high. I hear people say they want to see the human element of the game remain, and I do as well, but I want my human element to be the play on the field, not the way it is umpired--I want less of that.

Instant replay is being added to assist in making correct calls, since Major League Baseball made the decision that the right call is more important than tradition. Likewise, if the technology exists to give precise pitch location in real time, it can be used to make ball and strike calls. No one enjoys a game decided on an incorrect strike three or ball four, and it would be an interesting study to see how many games ended on an incorrect call. I'm sure the number is small, but any game is too many--mistakes should never be the deciding factor in a game, especially when those mistakes can be eliminated.

The current contract between Major League Baseball and the umpires union expires at the end of 2014 and is probably in the process of negotiation as we speak. PITCHf/x data was fairly new at the time of the last contract, but is established today. I'm not suggesting umpires be eliminated, and moving to a system of automated balls and strikes would not diminish the game.

Chances are, this technology is already incorporated into umpire evaluation. There is a decrease in "bad" calls in this brief time span, suggesting umpires are striving to make better calls. In the end, it doesn't matter--no one ever walked out of a Major League stadium saying "Man, that was one of the best called games by a home plate umpire I've ever seen!". A personal pet peeve of mine has always been the concept of the individual strike zone, something which suggests an element of subjectivity each umpire brings to the plate. Calling balls and strikes adds nothing to the game and removing it would generate about a week of outrage until people realize how little they miss it.

It's going to happen sometime--the bottom of the net was cut off in basketball, substitution was allowed in football, and the designated hitter arrived in baseball. The Brewers and Astros even changed leagues--change can happen. Why not now, and move from getting calls right 85% of the time to 100%, with the side benefit of eliminating arguments over balls and strikes and speeding the game along?

Special thanks to Daren Willman at baseballsavant.com for all the data. Any errors in collecting, collating or representing the data are mine alone.

Scott Lindholm is a web columnist for 670 The Score in Chicago. Follow him on Twitter at @ScottLindholm.