The Dodgers are elite defensively. Or maybe just average. We’re not sure.

The game of baseball is like a map of the world. We have accurate, detailed maps of every speck of land on the planet. Anyone can go on Google Maps and zoom in to a 10 ft.=1 in. scale satellite image of pretty much anywhere on Earth. There are still some fuzzy areas in places like Havana and Pyongyang, but for the most part, we’ve got the green areas figured out. Similarly, we have a pretty good understanding of hitting. There’s always more to explore, but we have a solid idea of who is and who isn’t a good hitter.

However, more than half of the Earth is covered in water. We have some idea what the planet looks like under the oceans, but much is unknown. According to NASA, “Even with all the technology that we have today— satellites, buoys, underwater vehicles and ship tracks— we have better maps of the surface of Mars and the moon than we do the bottom of the ocean.” This is a lot like player defense. Even with all the tracking software and batted ball data at our disposal, we’re still not too far advanced from the old “eyeball test.”

Take the 2019 Dodgers for example. By Defensive Runs Saved (DRS), they are far and away the best defensive team in baseball. Their collective 123 DRS through Saturday’s games leads the second place Diamondbacks by 37. In fact, 123 DRS is the second most by any team since 2003— as far back as we have data— trailing only the 2018 Diamondbacks (157).

By Ultimate Zone Rating (UZR) though, they are exactly league average, ranking 16th in MLB with -0.5. These are the two best metrics we have available for team defense, yet they disagree by an astounding amount of runs. To find the reason for the disparity, we need to look closer at what each stat is— and isn’t— measuring.

DRS

Developed by The Fielding Bible, DRS is calculated on a plus/minus system. Using information manually inputted by Baseball Info Solutions, each batted ball is given a plus rating if the play is made by the defender or a minus if it isn’t. How much is added or subtracted varies depending on how hard the ball was hit, what type of hit it was (ground ball, line drive, or fly ball), and location.

Here’s an example from The Fielding Bible’s overview:

The computer totals all softly hit ground balls on Vector 17, for example, and determines that these types of batted balls are converted into outs by the shortstop only 26% of the time. Therefore, if, on this occasion, the shortstop converts a slowly hit ball on Vector 17 into an out, that’s a heck of a play, and it scores at +.74. The credit for the play made, 1.00, minus the expectation that it should be made, which is 0.26. If the play isn’t made—by anybody—it’s -.26 for the shortstop.

DRS isn’t flawless, though. It’s based on human inputs from Baseball Info Solutions, so there will be human error. It’s also based on positional averages— first basemen are compared to first basemen and shortstops to shortstops— but doesn’t account for first base being an easier position to play than shortstop. Furthermore, there’s no account for the important nuances of catcher defense, namely pitch framing.

There are actually ten different components of DRS, but the most important for these purposes might be the most recently added:

10. Shift Runs Saved – Measures team’s ability to defend when using a defensive shift

According to Baseball Savant, the Dodgers shift more than any other team in baseball. They’ve used a shift on 50 percent of all plate appearances this season. Given that team shifting is incorporated into DRS, this might be the most accurate measure of what’s really happening in Los Angeles.

UZR

Like DRS, UZR is also based on Baseball Info Solutions data. Mitchell Lichtman is the architect of the metric, and he provides a thorough explanation at FanGraphs of how it’s calculated. Essentially, it looks at multiple years of batted ball data and determines a percentage for whether each batted ball will be fielded. It then compares these percentages to the actual outcome of each play, and assigns run values accordingly.

Let’s say that that same batted ball in the example above was caught by the CF’er on the first play of a game. Since typically someone will catch that same ball only 25% of the time (see above), this particular CF’er will get credit for an extra .75 plays – 100% minus 25%. We then convert .75 plays into runs by multiplying .75 by the difference between an average hit in that location and the average value of an air ball out. A typical outfield hit is worth around .56 runs and any batted ball out is worth around -.27 runs, so the difference between a hit and an out is worth around .83 runs. (We don’t vary the value of the hit or out based on the outs or base runners because we want “game situation-neutral” defensive evaluations.) Since our fielder gets credit for .75 extra plays, we give him credit for .75 times .83 runs, or +.6255 runs for that play.

There are several other components to UZR as well, such as outfield arm and park adjustments. FanGraphs’ overall “DEF” measurement is based on UZR, though it adds in components like catcher framing and positional adjustments.

There also drawbacks; just like DRS it doesn’t account for positional difficulty or catcher framing. In fact, UZR is not available for catchers or pitchers at all. DRS uses smaller buckets to measure batted balls, though Lichtman contends that larger is better. “I don’t know that they do, but if they use distance and vector, say in degrees, that is a lot of buckets and if they are not using a “smoothing” function or some such adjustment, they are asking for trouble. You simply CANNOT use that many buckets without a smoothing function. And if you are using a smoothing function, you ALMOST might as well just use larger (fewer) buckets.”

Getting back to the 2019 Dodgers, UZR makes a critical omission that penalizes them:

The data includes whether a shift (a generic one, in the opinion of the “stringer” – the person recording the data) was on, and whether the shift likely affected the play at all. If it did – again, according to the “stringer”- then the play is ignored.

Given that the Dodgers shift on 50 percent of all plays, UZR is only measuring the other half. That alone could account for an enormous difference in value— maybe even 123.5 runs worth!

Results and other stats

It certainly appears as though the exceptionally favorable view of the Dodgers from DRS is probably the most accurate. Baseball Prospectus’ Park Adjusted Defensive Efficiency appears to corroborate this finding, ranking the Dodgers second behind the Astros with 3.63. (For the record, the Astros shift 49.5 percent of the time— just a hair less than the Dodgers.)

That’s not to say DRS is always better than UZR. They both have their uses and drawbacks. If you’re looking at catcher data, Baseball Prospectus’ Catcher Defensive Adjustment is probably your best bet, as it incorporates framing, blocking, and arm at appropriate weights.

This could all be moot in a short while anyway. Some of the best defensive metrics are coming from Baseball Savant. Their Outs Above Average is based on Statcast rather than human input, so it’s likely to be more precise than either DRS or UZR. Unfortunately, they don’t have data for infielders or team stats yet (though they do have catcher framing). Once they develop a more complete set of Statcast-based defensive metrics, they will likely become the go-to for defense valuation.

In the meantime, the Dodgers appear to be one of the best defensive teams ever. At least that’s what our limited ocean floor-mapping technology reveals, anyway.

Daniel R. Epstein is an elementary special education teacher and president of the Somerset County Education Association. Tweets @depstein1983.