Quantifying the Impact of Defensive Uncertainty

Recently in the sabermetric community there has been a lot of discussion about fielding stats and their inclusion in WAR (see for example this thread, or this one at The Book blog) given the uncertainty behind the data (batted ball type, hit location etc.). With that in mind I thought it would be an interesting exercise to see how applying uncertainty to the defensive runs above average (DRAA) numbers affects the 2009 fWAR leaderboard. My method for applying the uncertainty is pretty simple; I just ran a Monte Carlo simulation using a normal distribution for the simulated DRAA with a mean of the DRAA reported by Fangraphs and a standard deviation of 5 runs. The following table looks at how often the top 10 players in fWAR fell into each of the top 10 slots after running the simulation 10000 times.

 

1 2 3 4 5 6 7 8 9 10
Albert Pujols 62% 23% 9% 4% 1% 0% 0% 0% 0% 0%
Ben Zobrist 22% 36% 22% 11% 5% 2% 1% 1% 0% 0%
Joe Mauer 12% 24% 29% 17% 9% 5% 2% 1% 0% 0%
Chase Utley 3% 8% 16% 23% 19% 13% 9% 5% 3% 1%
Derek Jeter 1% 4% 10% 16% 18% 17% 13% 9% 5% 3%
Hanley Ramirez 0% 2% 7% 12% 16% 18% 16% 11% 8% 5%
Evan Longoria 0% 2% 4% 10% 14% 17% 17% 14% 9% 6%
Prince Fielder 0% 0% 2% 5% 8% 12% 15% 16% 15% 10%
Ryan Zimmerman 0% 0% 1% 2% 4% 7% 11% 15% 16% 15%
Adrian Gonzalez 0% 0% 0% 1% 3% 6% 8% 13% 16% 15%

 

So if you buy my 5 run SD assumption then the impact on ordinal ranking is the above.  Clearly the impact on overall WAR (and thus $/WAR) isn't captured in the above analysis.

 

This is just a quick look at the subject, but I think there may be more to uncover like looking at different fielding metrics in place of UZR.  Either way it answered one of my questions, "What orders of magnitude are we talking about?"

Update:  Here's the same table with a SD of 10 runs

1 2 3 4 5 6 7 8 9 10
Albert Pujols 38% 21% 14% 9% 6% 4% 3% 2% 1% 1%
Ben Zobrist 22% 20% 16% 11% 9% 6% 4% 3% 2% 2%
Joe Mauer 15% 17% 15% 12% 10% 8% 6% 4% 3% 2%
Chase Utley 8% 11% 12% 11% 10% 9% 8% 6% 5% 4%
Derek Jeter 5% 8% 10% 10% 10% 9% 8% 7% 6% 5%
Hanley Ramirez 4% 7% 8% 9% 10% 9% 8% 7% 6% 5%
Evan Longoria 3% 6% 7% 9% 9% 9% 8% 7% 7% 5%
Prince Fielder 2% 4% 5% 7% 7% 8% 8% 7% 7% 6%
Ryan Zimmerman 1% 2% 4% 5% 6% 7% 7% 7% 7% 6%
Adrian Gonzalez 1% 2% 3% 5% 5% 6% 6% 7% 6% 6%
X
Log In Sign Up

forgot?
Log In Sign Up

Forgot password?

We'll email you a reset link.

If you signed up using a 3rd party account like Facebook or Twitter, please login with it instead.

Forgot password?

Try another email?

Almost done,

Join Beyond the Box Score

You must be a member of Beyond the Box Score to participate.

We have our own Community Guidelines at Beyond the Box Score. You should read them.

Join Beyond the Box Score

You must be a member of Beyond the Box Score to participate.

We have our own Community Guidelines at Beyond the Box Score. You should read them.

Spinner.vc97ec6e

Authenticating

Great!

Choose an available username to complete sign up.

In order to provide our users with a better overall experience, we ask for more information from Facebook when using it to login so that we can learn more about our audience and provide you with the best possible experience. We do not store specific user data and the sharing of it is not required to login with Facebook.

tracking_pixel_9351_tracker