Graph of the Day: Comparing Power Rankings

Comparing_power_rankings_medium

Power Rankings are funny.  They all try to measure something different and cater to a different audience.  And no matter the system, fans of some team will disagree vehemently.  Nonetheless, i was curious to see how Justin's BtB rankings compared to other systems.

The comparison set: Pythagorean wins, second- and third-order wins from BPro, BPro's Hit List factor (which is simply an average of actual record, Pythag, and second- and third-order winning percentage), and RPI.  BtB_lg removes the league adjustment.

All systems are compared to actual records as of late last week using both correlation and total absolute error.  Now, "error" isn't a bad thing here, because these power rankings aren't trying to predict past record, they're trying to extract true talent and ignore fluke.

In general, as correlation goes up, aboslute error should go down.  One system where that doesn't happen is RPI, which correlates the closest to actual record, but has a much tighter set of ratings, making its absolute error quite high.  The system that matched actual records the least was BtB's, followed by the non-league-adjusted BtB number and the two most adjusted BPro systems.  Personally, I'd like to see a power ranking NOT match actual record that closely, because I want new information.  Of course, not matching closely doesn't mean the system is any good.

Anyway, my major takeaways from this graph are that BtB's system definitely doesn't just match win-loss record (as many commenters have creatively pointed out) and that RPI and BPro's systems DO.

X
Log In Sign Up

forgot?
Log In Sign Up

Forgot password?

We'll email you a reset link.

If you signed up using a 3rd party account like Facebook or Twitter, please login with it instead.

Forgot password?

Try another email?

Almost done,

Join Beyond the Box Score

You must be a member of Beyond the Box Score to participate.

We have our own Community Guidelines at Beyond the Box Score. You should read them.

Join Beyond the Box Score

You must be a member of Beyond the Box Score to participate.

We have our own Community Guidelines at Beyond the Box Score. You should read them.

Spinner

Authenticating

Great!

Choose an available username to complete sign up.

In order to provide our users with a better overall experience, we ask for more information from Facebook when using it to login so that we can learn more about our audience and provide you with the best possible experience. We do not store specific user data and the sharing of it is not required to login with Facebook.

tracking_pixel_9351_tracker