Graph of the Day: Comparing Power Rankings
Power Rankings are funny. They all try to measure something different and cater to a different audience. And no matter the system, fans of some team will disagree vehemently. Nonetheless, i was curious to see how Justin's BtB rankings compared to other systems.
The comparison set: Pythagorean wins, second- and third-order wins from BPro, BPro's Hit List factor (which is simply an average of actual record, Pythag, and second- and third-order winning percentage), and RPI. BtB_lg removes the league adjustment.
All systems are compared to actual records as of late last week using both correlation and total absolute error. Now, "error" isn't a bad thing here, because these power rankings aren't trying to predict past record, they're trying to extract true talent and ignore fluke.
In general, as correlation goes up, aboslute error should go down. One system where that doesn't happen is RPI, which correlates the closest to actual record, but has a much tighter set of ratings, making its absolute error quite high. The system that matched actual records the least was BtB's, followed by the non-league-adjusted BtB number and the two most adjusted BPro systems. Personally, I'd like to see a power ranking NOT match actual record that closely, because I want new information. Of course, not matching closely doesn't mean the system is any good.
Anyway, my major takeaways from this graph are that BtB's system definitely doesn't just match win-loss record (as many commenters have creatively pointed out) and that RPI and BPro's systems DO.
0 recs |
8 comments
Comments
Sweet!
My system is the least close to reality of all the systems!!
The reason, of course, is source data. Other systems are using win or run totals, and we’re estimating those data from more fundamental data. I think that’s why I like this system. If your first look at team performance are true team records, and the second look is Pythagorean record, these rankings provide a third, novel look at the same teams.
-j
My blog: Basement-Dwellers.com
by JinAZ on Jun 24, 2009 4:42 PM EDT reply actions 0 recs
First off, what is RPI?
Secondly, it doesn’t seem to make sense to compare it to actual records because those aren’t really what you want out of a power ranking. Justin’s seems to be the most thorough, I would say it’s probably a bit better than 3rd Order Wins because it values pitching using tRA instead of just reverse EqA.
Also, and I don’t know how easy this would be to do, but wouldn’t it be better to compare it to a full seasons record? You could use each formula and calculate final rankings using last years teams, then compare those to the final record.
St. Louis relievers... defying win expectancy since 2008
http://www.drivelinemechanics.com/
by vivaelpujols on Jun 24, 2009 6:40 PM EDT reply actions 0 recs
I agree with all that.
I wasn’t comparing the rankings to actual record to decide how good they were. I was comparing them to see how much information other than actual wins they contained.
Beyond the Boxscore Not a member? Sign up.
by Sky Kalkman on Jun 25, 2009 12:21 AM EDT up reply actions 0 recs
"Correl" (whatever it is) is bad, right?
I'm not a sabermetrician, but I do play one at Driveline Mechanics.
by devil_fingers on Jun 24, 2009 7:48 PM EDT reply actions 0 recs
No It's good
It stands for correlation which means “fit”.
St. Louis relievers... defying win expectancy since 2008
http://www.drivelinemechanics.com/
by vivaelpujols on Jun 24, 2009 9:40 PM EDT up reply actions 0 recs
I guess I should hvae used the /sarc
I'm not a sabermetrician, but I do play one at Driveline Mechanics.
by devil_fingers on Jun 24, 2009 11:38 PM EDT reply actions 0 recs
crap
St. Louis relievers... defying win expectancy since 2008
http://www.drivelinemechanics.com/
by vivaelpujols on Jun 25, 2009 1:18 AM EDT up reply actions 0 recs

by 














BtB on Facebook














