How Well Does the Baseball America Prospect Top 100 Estimate the Top 100 Future Players?

Introduction

The 2013 MLB draft is nearly upon us and what does that mean? More players eligible for top 100 prospect lists! The purpose of these lists is to find which players will be among the top future major league players. How well has Baseball America done with this? Enter Fangraphs WAR to find the answer.

Method

My method was to take Baseball America's top 100 lists and compare them to the top 100 players by fWAR each year, mixing hitters and pitchers together.

The first question that needs to be answered is how many years into the future should we look when viewing a prospect list? Obviously some players are closer to the majors than average. Instead of looking at every time a player was ranked, I decided to only look at the last time a prospect was ranked, since this must be the closest they are to the majors.

The best way to find when a prospect has matured is to look at how many years into their career the top 100 players are historically. I've also included the average WAR for those players over the years as well. I started from 1988 just to give some context before the lists start in 1990.

(Click to enlarge)

There are expectedly big bumps in the road in the strike-shortened 1994 season. The average career length of the top 100 players jumped that year, but the younger-than-average Jeff Bagwell, Frank Thomas, Ken Griffey Jr., and Kenny Lofton were all in the top five.

There has been a youth movement since 2005. The average distance into a player's career for the top 100 has dropped from over six in the mid-2000s to the low fives, with a low of 4.84 in 2010. Over that time, the average WAR values for the top 100 has dropped from about five to the high fours.

When Do Prospects Break Out?

How long should you wait before considering a top prospect a bust?

The top few spots tend to be taken by well-established players. The average distance into a career for the top player is over eight years and the average career length drops near the back of the top 100. Overall this tells us that we should give about six years for a player to fully mature after his last top 100 prospect appearance.

There are 936 players who are eligible (removing duplicate listings and with a 6th year occurring before 2013). 397 of those players (42%) were not even in the major leagues in their 6th year. Of those who were, the average WAR was 1.7, with a range from -1.9 to 11.9. The best 6th year players were Pedro Martinez, Adrian Beltre, Chase Utley, Jeff Bagwell, and Albert Pujols.

Results

How many players in the top 100 of WAR each year were ranked by Baseball America?

(Click to enlarge)

The first six to 10 years after BA put out their first list, the majors were building up with the prospects of the late 80s and early 90s (John Olerud! Larry Walker! Cullen Hartzog!) . Since 2004, about 70% of the top 100 players in the major leagues were top 100 prospects at some point in time, which is frankly pretty good. Now look at how these percentages stack up by bins of 20 (Top 20 means the top 20 players in the majors that year). I used a 5-year running average:

(Click to enlarge)

It looks like BA has gotten better at finding the elite prospects in recent years. The largest percentage of top 100 players ended up as 21-40 types instead of top 20, but that trend has switched since the mid 2000s.

How much do the top 100 prospect lists tell us about the player's likelihood six years from now? To start, only 58% of prospects are still in the major leagues in their 6th year.

Let's compare the 2006 prospect list with the 2012 actual top 100 players. Numbers one through five on the 2006 list were Delmon Young (-0.9 WAR), Justin Upton (2 WAR), Brandon Wood (Not in majors), Jeremy Hermida (0.2 WAR), and Stephen Drew (-0.3 WAR). The top five players in 2012 were Mike Trout (#3 on 2012 list), Robinson Cano (Never ranked), Ryan Braun (#49 in 2006), Buster Posey (#7 in 2010), and David Wright (#21 in 2004).

So perhaps just looking at six years out is a bad idea. Most of the top players were prospects, but none of them were on the list six years previous. So instead of looking at a specific year, how does the final prospect ranking compare with the top ranking of WAR in a player's career? On average, there are 1,100 players per season, but only the top 200 are above average. The rest have WARs below two. For this chart, I included a 10-period moving average.

(Click to enlarge)

That's actually a pretty nice trend. Now a related chart, but put differently. How does the final ranking for a prospect relate to their average career WAR? Again, this includes a 10-period moving average, along with a best fit line:

(Click to enlarge)

That's a similar trend with a similar correlation value, but this one fits best as a third-order equation. There is a big drop off after the top 20 players, which is something I have noticed previously. The top 20 prospects are the elite ones and everyone else is just trying to catch up.

But all of these charts forget one thing: not all players make the major leagues. How does a prospect's final ranking relate to the likelihood of him appearing in the major leagues? Here are those results as a 10-period moving average:

(Click to enlarge)

Players whose last ranking is in the top 10 have always made an appearance in the major leagues. The amount of players without an appearance rises steadily throughout the top 100, especially after the first 50.

What about a faux-aging curve? I didn't do any weighting by plate appearances, just took the average WAR of all ranked players for a given number of years after their last appearance on a top 100 list:

(Click to enlarge)

Obviously, there's a selection bias happening after 10 years. Only the good prospects are still around, so their average WAR is higher than we would expect based on the trend. This fits with known aging curves, though. Most prospects are in their early 20s in their final appearance on a list, so a peak WAR around 26 or 27 passes the sniff test.

Finally, let's look at how many prospects exceed certain thresholds in increasing numbers of years after they were last ranked. How many were in the major leagues, how many were average (2 WAR), all-stars (4 WAR), and MVPs (6 WAR)?

(Click to enlarge)

The most players appeared two seasons after they were last ranked. MVP seasons for prospects most often occur three years into their career, but the highest amount can occur anywhere from two to seven years after their final ranking.

Summary

I threw a lot of graphs out there, but the biggest takeaway is the second one: nearly 80% of the top 100 players in recent seasons were ranked in the top 100 by Baseball America. Of all the talk about failure rates (12% of prospects don't even get to the majors, only 69% fail to put up a career WAR over 10), the fine folks at BA are really good at finding the top players.