Comparision of UZR/150 Projections

Recently, Steve Sommer (click here for link) and I (click here for link) both released projected UZR/150 values for the 2010 season.

Note: When I published the original data, I didn't have the UZR/150 rates, but I now have them included.

Steve used a regressed three year weighted average of UZR along with a one year weighting of Tom Tango's Fan Scouting Report (FSR). I extended my values back 4 years, but did not include FSR data. Steve and I both decided to compare our data and run our separate analysis in 2 separate articles (link to Steve's article).  

I don't believe just using UZR values is the best method, but it was a start for me. I planned to attempt to add FSR to my projections next year, but I figured I would baby-step my way into the projection realm. Seeing that Steve also made the attempt, I wanted to see how our data compare and what can be done to further improve it.  Here are three areas where the results showed some useful information or where further analysis could be done.   

Aging Factor

On average, Steve's UZR/150 values were 0.16 higher than mine. I expected the value to be higher since I had a -0.7 UZR/150 aging factor added onto my values and he didn't use an aging factor. This tells me that the fans have already observed and factored into their values the player's aging process. I am not for sure about the accuracy of the -0.16 UZR/150 aging factor, but it will give anyone using the FSR for projections a reasonable starting point.

Standard Deviation Between Datasets.

The overall standard deviation (SD) of the difference between Steve's and my values was 2.0 UZR/150. This puts 95% of all the values within ~4 UZR/150. By looking a little farther, the more a player played at a position, the lower the standard deviation. For example players with <150 games at the position over the last 4 seasons had a standard deviation of 2.4 UZR/150 while the players that had over 600 games at the position had a SD of 1.3 UZR/150. Here is a table that summarizes the results:

Not much to say that isn't obvious. The more a player plays, the more his talent is revealed.

Outliers

Here are the players whose values deviated by more than 2 SDs (with Everett and Cust being the only 2 that varied by more than 3 SD):

Again, many of these players have not spent much time in the major leagues. The average number of games for all 317 players was 279 games at the position over the last 4 years while the players on the previous list averaged only 182 games.

Two players had more than 279 games at their position: Adam Everett and Austin Kearns.  The fans considered them worse at their position than just UZR/150 would indicate. Neither player is a offensive power, and while two people is not a significant sample size at all, it would be interesting to see if or if not the player's offensive ability sways the FSR voters.

Conclusions

I happy with how close the results were and in the future I would like to incorporate the FSR into my data. I would probably need find an aging value also.  Finally, as previously stated, I would like to see if the FSR is biased to towards offensive players getting better grades from the fans.

I enjoyed this endeavor with Steve and hopefully we can work on more projects in the future. I know that defensive statistics aren't close to an exact science, but with some more understanding a person may not end up in a heated discussion on Coco Crisp's defense.