Past Performance Does Not Guarantee Future Gains
I know faithful BtB readers would never fail to discount attrition risk or regress expectations to the mean. I know this because you're a smart group of people. But I also tend to accept the presumption that cognitive biases sneak up on us when we're least cautious. And the more "enlightened" we come to think of ourselves, the more we might potentially open ourselves to blind spots.
Take free agent valuations. These things aren't easy, especially when you consider that few of the publicly available prediction models have released projections for the 2010 season. So much of the commentary you see out there is relying on guess work. And can you fathom when cognitive biases might play their most prominent role? Could it perhaps be when the analysis is unmoored from rigorous statistical data?
Take Pedro Feliz (that's "Pete Happy" in English). The Phillies held a team option on his services for the 2010 season: they could exercise it and pay him $5 million to stay in the City of Brotherly Love, or they could decline the option and pay him a $500,000 buyout. Thus, the marginal cost of a year of sterling defense and head-desk inducing limp noodleism was $4.5 million. Yesterday, the Phillies declined Feliz's option and will likely seek to fill their hot corner needs from the hot stove.
How can we tell if this was a good move?
One place many have begun their analysis is FanGraphs. There, they have calculated each player's value in terms of WAR. They have also taken the extra step of multiplying each season's WAR figure by the price of a marginal win on the free agent market in a given year to get a sense of how much each player was "worth."
Here are Feliz' values for the previous five seasons:
| Team | WAR | Value | |
| 2005 | Giants | 2.6 | $8.70 |
| 2006 | Giants | 1.9 | $7.00 |
| 2007 | Giants | 2.8 | $11.50 |
| 2008 | Phillies | 1.5 | $6.70 |
| 2009 | Phillies | 1.3 | $5.70 |
And because I know you like a good graph:
As you can see, Feliz has been in decline. However, he has never been worth less than $4.5 million in any season in the last five years. Thus, the Phillies were wrong to decline his option, correct?
Well, no. First, we have to come up with an estimate of Feliz' value if he plays 162 games. Then we have to discount that figure by the probability that he gets injured. Playing next season at age 35, and with this probability is increasing year-over-year at a faster rate than it ever has, it is likely that there is at least a 10% chance Feliz spends a month or more on the disabled list. There is also a non-zero chance he suffers a career ending injury, etc.
So if our original estimate is that Feliz' value will be approximately $5 million in the event that he is healthy, it is almost certainly true that the Phillies made the right decision in declining his option.
This conclusion is reinforced once we take the necessary step of evaluating the other options. The two most commonly discussed are Chone Figgins and Adrian Beltre.
Here's a typical discussion of Figgins, who is rather difficult to value:
So with all this information at our disposal, I ask – what’s the value of Chone Figgins? Since 2007, FanGraphs calculates that he has been worth a total of $50.9 million, with a high of $27.4 million in 2009 (while getting paid a paltry $5.78 million). But with Figgins finally eligible for free agency, would you pay Figgins $50 million over the next three years?
Figgins' FanGraphs values are even more all over the map:
| Team | WAR | Value | |
| 2005 | Angels | 2.7 | $9.20 |
| 2006 | Angels | 0 | $0.10 |
| 2007 | Angels | 3.1 | $12.50 |
| 2008 | Angels | 2.4 | $11.00 |
| 2009 | Angels | 6.1 | $27.40 |
And here's a graph:
Eyeballing that chart, you are likely willing to discount his outlier season of 2009, despite the fact that it is the most recent. It was fueled largely by a career high 16.3 fielding runs above average (by UZR), and perhaps you don't trust UZR's year to year fluctuations (motivated by good reasons).
So let's say you throw out his outlier 2006 and 2009 seasons, and expect him to be worth between $9 million and $12 million. We'll split the difference and say $10.5 million. Again, that is if he is healthy. He will play next season at age 32, so perhaps the risk there is lower than it is with Feliz.
But does the bigger fluctuation in value (think of a beta value) make Figgins worth less than his expected value? Possibly. If teams are looking for production certainty, they may be willing to pay a premium. However, we ought to wonder whether the fluctuations are related to something inherent about Figgins, or are simply a combination of luck and circumstances.
In any event, if the Phillies are going to pay what is essentially par for the third baseman, perhaps they would like to guarantee a higher level of production than that which Feliz can be counted upon to provide.
The important thing is not to allow the ready availability of FanGraphs numbers, which are evaluated in hindsight, color your expected valuations. To do so is to fall prey to a common set of cognitive biases.
And, of course, if they want to make the ballsy play, they'll go after the riskiest but also probably the best option: Adrian Beltre.
0 recs |
11 comments
| Add comment
|
Comments
On UZR's purported unreliability
http://www.fangraphs.com/blogs/index.php/uzr-2008-to-2009/
So the lesson is, when there’s not a lot of UZR data on a player, there will be a lot of noise, but as the sample size increases, the data (at least from 2008 to 2009) actually becomes almost as highly correlated year to year as the stats that are considered to be the most reliable. (like wOBA)
by Crashburn Alley on Nov 9, 2009 12:40 PM EST reply actions 0 recs
Yeah, I agree with that assessment.
But I do think the park adjustments are poor.
by Tommy Bennett on Nov 9, 2009 12:41 PM EST up reply actions 0 recs
For example
I am not sure Feliz’ defensive decline from ‘06 to ’07 isn’t attributable mostly to differences in home park.
by Tommy Bennett on Nov 9, 2009 12:46 PM EST up reply actions 0 recs
Nagging injury?
Aaron King is still my homeboy... iffy mechanics and all
McFAQ for all you newcomers out there.
GET THAT VORP AND WHIP SH!T OUTTA HERE!!!
by baetown415 on Nov 9, 2009 3:55 PM EST up reply actions 0 recs
The answer is a bit murkier than that.
The issue is playing time. The average qualified starter gets 270 expected outs a season or so (what I presume is being referred to as UZR chances there). A qualified starter gets about 600 PAs, to contrast.
Also, remember that you have a smaller spread in UZR than you do in offensive metrics due to selective sampling (well, that’s not the ONLY reason, but it’s certainly a big reason and the one most relevant to this discussion). In other words, if we were to similarly break out wOBA relative to position we’d probably see the correlation incease as well, because the variance would be smaller.
UZR is unreliable. It is more unreliable than wOBA (which is, it should be noted, also somewhat unreliable). Defense also has a very different looking aging curve from offense, which I think some people forget. (Defense peaks basically at debut.)
And here’s a little brain teaser for everyone. When regressing UZR to the mean, what mean should we use? For instance, let’s say you have a -11 UZR/150 player at shortstop. Do you expect him to regress to the mean of all shortstops, and improve? Or to regress to the mean of all MLB players, and decline (or more likely, be moved off the position altogether?)
by cwyers on Nov 9, 2009 1:18 PM EST up reply actions 0 recs
My response to the regression question is always position mean, but the question is definitely valid
I don’t know or think that my answer is correct.
Marlin Maniac, a Florida Marlins blog
Come attend Intro to Sabermetrics 101!
Check me out at Beyond the Box Score as well.
by SFiercex4 on Nov 9, 2009 1:39 PM EST up reply actions 0 recs
It's an interesting question.
The easiest answer is, as you say, to regress to the position mean. It will appear to be the most “correct” – in other words, players that play in consecutive seasons will regress toward the positional mean.
Where this gets interesting is that you are, in fact, only looking at players able to stay at the position. Let’s consider, for a moment, our poor defensive shortstop again. If he is truly a decent enough shortstop, he will stay at the position (there are other factors, such as offense, to consider as well – I’m simplifying a bit). If he is not, he will likely either be moved to second or third, or simply not play in the majors the following season, depending on how good his bat is.
So if we want to answer the question of, “how good a defensive shortstop is this player?” then regressing to position seems correct.
But if we are trying to answer a question as posed in this article, are we only interested in projecting (say) projecting Figgins’ defense in that way?
by cwyers on Nov 9, 2009 2:33 PM EST up reply actions 0 recs
Is this a question that could be answered by the data?
I’d assume one regression does a better job at forecasting than the other.
My intuition tells me you should regress to the mean of all players, the reason being that you do the exact same thing with other statistics. You don’t regress wOBA on a positional basis, which is a measure of one’s offensive talent. Why do that with defensive talent? You’re basically regressing to a sub-group of players with a “similar” ability level.
by shawndgoldman on Nov 9, 2009 2:25 PM EST up reply actions 0 recs
You're comparing different types of statistics here.
UZR is more like a central moment in that it is a comparison of each value against some mean.
A shortstop with UZR = 0 is different from a first baseman with a UZR = 0. Putting the first baseman at short would result in a much lower UZR — the positional adjustments try to capture this aspect with a simple adjustment that says that a shortstop of UZR = 0 is more like a UZR of 5 when compared against the “average” player, while a 1B with a UZR = 0 is more like a UZR of -8 when compared against the “average” player.
wOBA is a general moment — it is not centralized, and therefore rather than being compared to zero, it is compared to a league average of ~.330 – so regression to the mean, in this case, is straight-forward. There’s no +5 runs given to a 1Bs offense over SS as each single, walk, or home run counts the same.
As such, I’d argue that UZR needs to be treated on a positional basis since the stat itself is dependent upon the performance of others at that position – or at worst regressed to the mean AFTER positional adjustments; while wOBA is calculated independent of position.
by Trickman on Nov 9, 2009 2:41 PM EST up reply actions 0 recs
I don't know if it's a question that can be definitively answered at all, at least not as posed.
The correct answer is probably “it depends on what question you are attempting to answer with the data.”
And yes, due to the selective sampling issue, one is always going to do better at predicting what is observed – in this case, regressing to position mean is always going to be more accurate than regressing to league mean, looking only at players who play the same position in consecutive seasons. But does being more accurate in that narrow sense make it more “correct?” Again, there is not one good answer for this.
A similar issue arrises when it comes to projecting players with little to no major league experience. All projection systems I am aware of regress translated stats to the MLB mean. And that is going to provide, in retrospect, the best projections of those players who played in MLB. But does it do a very good job of projecting what rookies are ready for MLB, and which aren’t?
by cwyers on Nov 9, 2009 2:47 PM EST up reply actions 0 recs
Here's where regressing to a supposedly "unbiased" scouting report would really help
MGL was talking about this last week. Then again, where are you going to find such a report and how are you going to convert it to a meaningful number?
Marlin Maniac, a Florida Marlins blog
Come attend Intro to Sabermetrics 101!
Check me out at Beyond the Box Score as well.
by SFiercex4 on Nov 9, 2009 4:39 PM EST up reply actions 0 recs

by 












BtB on Facebook
















