Recently in the sabermetric community there has been a lot of discussion about fielding stats and their inclusion in WAR (see for example this thread, or this one at The Book blog) given the uncertainty behind the data (batted ball type, hit location etc.). With that in mind I thought it would be an interesting exercise to see how applying uncertainty to the defensive runs above average (DRAA) numbers affects the 2009 fWAR leaderboard. My method for applying the uncertainty is pretty simple; I just ran a Monte Carlo simulation using a normal distribution for the simulated DRAA with a mean of the DRAA reported by Fangraphs and a standard deviation of 5 runs. The following table looks at how often the top 10 players in fWAR fell into each of the top 10 slots after running the simulation 10000 times.
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | |
|---|---|---|---|---|---|---|---|---|---|---|
| Albert Pujols | 62% | 23% | 9% | 4% | 1% | 0% | 0% | 0% | 0% | 0% |
| Ben Zobrist | 22% | 36% | 22% | 11% | 5% | 2% | 1% | 1% | 0% | 0% |
| Joe Mauer | 12% | 24% | 29% | 17% | 9% | 5% | 2% | 1% | 0% | 0% |
| Chase Utley | 3% | 8% | 16% | 23% | 19% | 13% | 9% | 5% | 3% | 1% |
| Derek Jeter | 1% | 4% | 10% | 16% | 18% | 17% | 13% | 9% | 5% | 3% |
| Hanley Ramirez | 0% | 2% | 7% | 12% | 16% | 18% | 16% | 11% | 8% | 5% |
| Evan Longoria | 0% | 2% | 4% | 10% | 14% | 17% | 17% | 14% | 9% | 6% |
| Prince Fielder | 0% | 0% | 2% | 5% | 8% | 12% | 15% | 16% | 15% | 10% |
| Ryan Zimmerman | 0% | 0% | 1% | 2% | 4% | 7% | 11% | 15% | 16% | 15% |
| Adrian Gonzalez | 0% | 0% | 0% | 1% | 3% | 6% | 8% | 13% | 16% | 15% |
So if you buy my 5 run SD assumption then the impact on ordinal ranking is the above. Clearly the impact on overall WAR (and thus $/WAR) isn't captured in the above analysis.
This is just a quick look at the subject, but I think there may be more to uncover like looking at different fielding metrics in place of UZR. Either way it answered one of my questions, "What orders of magnitude are we talking about?"
Update: Here's the same table with a SD of 10 runs
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | |
|---|---|---|---|---|---|---|---|---|---|---|
| Albert Pujols | 38% | 21% | 14% | 9% | 6% | 4% | 3% | 2% | 1% | 1% |
| Ben Zobrist | 22% | 20% | 16% | 11% | 9% | 6% | 4% | 3% | 2% | 2% |
| Joe Mauer | 15% | 17% | 15% | 12% | 10% | 8% | 6% | 4% | 3% | 2% |
| Chase Utley | 8% | 11% | 12% | 11% | 10% | 9% | 8% | 6% | 5% | 4% |
| Derek Jeter | 5% | 8% | 10% | 10% | 10% | 9% | 8% | 7% | 6% | 5% |
| Hanley Ramirez | 4% | 7% | 8% | 9% | 10% | 9% | 8% | 7% | 6% | 5% |
| Evan Longoria | 3% | 6% | 7% | 9% | 9% | 9% | 8% | 7% | 7% | 5% |
| Prince Fielder | 2% | 4% | 5% | 7% | 7% | 8% | 8% | 7% | 7% | 6% |
| Ryan Zimmerman | 1% | 2% | 4% | 5% | 6% | 7% | 7% | 7% | 7% | 6% |
| Adrian Gonzalez | 1% | 2% | 3% | 5% | 5% | 6% | 6% | 7% | 6% | 6% |