Should teams use Baseball America's rankings to draft?

A few weeks after the 2013 Rule 4 draft, I published a research article regarding the monetary value of draft picks for major league teams. In that piece, I found that from 1990-2006 the gap in wins above replacement over the players first six seasons (the years of their rookie contract) between the actual draft picks and the players that "should have" been selected was quite large. In essence, teams in the draft have been leaving a ton of surplus value on the table because they still have not perfected the evaluation process. Granted, I am fully aware of the difficulties of scouting - it's as much an art as it is a science - but the gap is too large to simply ignore.

Thinking more about it, I wondered if teams wouldn't be better off just using the draft rankings compiled by Baseball America each year. Widely viewed as industry leader in draft coverage, BA has a network of sources contributing to all of their reports and rankings, meaning that the lists that they put out can be considered the closest thing to a consensus that's available. While lists like these have their detractors, there is undoubtedly value in getting multiple opinions from more than one organization. So, would teams benefit from just drafting off the Baseball America rankings?

To answer that, I first had to get the pre-draft rankings from Baseball America dating back as far as I could. Thanks largely to BA editor Matt Eddy (@MattEddyBA), I was able to find lists dating back to 2001, not nearly the sample size of the original study, but still a six-year window (2001-2006) to work with. From there, I used the same methodology as before, gathering six-year fWAR for the top-100 selections (excluding those players that did not sign) and for the top-100 ranked players according to Baseball America. I then grouped the selections into 6 tiers*: Tier 1 (top two picks), Tier 2 (picks 3-7), Tier 3 (picks 8-15), Tier 4 (picks 16-30), Tier 5 (picks 31-60), and Tier 6 (picks 61-100).

*Because the sample shrunk from 17 seasons to just six, I combined the top two picks into one tier, whereas I separated them in previous research. With more data I would have kept them apart.

Once again to quickly outline the tables below -- No 1-2 pick refers to the actual top two picks and BA 1-2 refers to the top two ranked players in the Baseball America rankings. The 6-year fWAR is an average of the six draft classes from 2001-2006, Market Value is found by multiplying the WAR by $4.92 million per win, Marginal Cost is found by taking 31% of the market value as outlined in our original study, and the Signing Bonus is an average of the top slot from the tier from the 2009-2012 drafts. All dollar amounts in the tables are in millions.

Tier 1

At the top, we see that in our six year sample teams would have been better served simply swapping their draft boards out for the BA rankings. Over that time frame, the top two ranked players actually out performed the actual selections by more than $12 million, total an extra three quarters of a win per season. The big player that was correctly identified as an elite talent by BA was Tim Lincecum. The magazine slotted Lincecum just behind Andrew MIller (oof!) in the 2006 draft class, but in reality he lasted until pick 10 because of questions regarding his size, durability, and unorthodox throwing mechanics.

Tier 2

In this tier we see the actual picks outperform the rankings, though we have to remember that part of that stems from the advantage in the first tier. Still, with a surplus of just over $7 million we can safely call this grouping a win for the actual clubs.

Tier 3

Tier 4

Tier 5

Tier 6

In our last four tiers we can hardly see any divide between the actual draft picks and the rankings. In fact, over the six-year period the top-100 draft picks totaled $86.24 million in net value and the BA lists totaled $89.53 million, a negligible difference. Of course, these results would be improved if we could add more years to each sample, but I doubt the results would favor one said too drastically in any event. Instead, it appears that major league baseball as a whole should continue to use their own boards and grading systems, though teams at the top might want to reference BA before making a pick. If we broke it down to an individual team basis, it's more likely that we would find teams that aren't doing as well as Baseball America is in their talent evaluation. Regardless, from these results I'd once again say that the best ways to build through the draft with the new CBA is by investing in getting the best evaluators and in turn finding better players in the latter rounds.

. . .

Special thanks to Chris St. John and Matt Eddy for help with this piece.

All stats courtesy of FanGraphs and Baseball-Reference.

Andrew Ball is a writer for Beyond the Box Score and Fake Teams.

You can follow him on twitter @Andrew_Ball.

More from Beyond the Box Score

Follow @BtBScore Follow @SBNationMLB

• Fun with pitching correlations
• What does it mean to be average?
• First and worst: Changeups of 2013
• Tim Lincecum and Learning to Help Yourself
• The Blue Jays should go for it, not rebuild in 2014