Beyond the Box Score: An SB Nation Community

Navigation: Jump to content areas:


Sports blogs for fans, by fans.
New Blog: RSL Soapbox for Real Salt Lake Fans!

Past Performance Does Not Guarantee Future Gains

Photo

More photos » by David J. Phillip - AP

I know faithful BtB readers would never fail to discount attrition risk or regress expectations to the mean. I know this because you're a smart group of people. But I also tend to accept the presumption that cognitive biases sneak up on us when we're least cautious. And the more "enlightened" we come to think of ourselves, the more we might potentially open ourselves to blind spots.

Take free agent valuations. These things aren't easy, especially when you consider that few of the publicly available prediction models have released projections for the 2010 season. So much of the commentary you see out there is relying on guess work. And can you fathom when cognitive biases might play their most prominent role? Could it perhaps be when the analysis is unmoored from rigorous statistical data?

Take Pedro Feliz (that's "Pete Happy" in English). The Phillies held a team option on his services for the 2010 season: they could exercise it and pay him $5 million to stay in the City of Brotherly Love, or they could decline the option and pay him a $500,000 buyout. Thus, the marginal cost of a year of sterling defense and head-desk inducing limp noodleism was $4.5 million. Yesterday, the Phillies declined Feliz's option and will likely seek to fill their hot corner needs from the hot stove.

How can we tell if this was a good move?

Star-divide

One place many have begun their analysis is FanGraphs. There, they have calculated each player's value in terms of WAR. They have also taken the extra step of multiplying each season's WAR figure by the price of a marginal win on the free agent market in a given year to get a sense of how much each player was "worth." 

Here are Feliz' values for the previous five seasons:

Team WAR Value
2005 Giants 2.6 $8.70
2006 Giants 1.9 $7.00
2007 Giants 2.8 $11.50
2008 Phillies 1.5 $6.70
2009 Phillies 1.3 $5.70

And because I know you like a good graph:

Feliz_medium

As you can see, Feliz has been in decline. However, he has never been worth less than $4.5 million in any season in the last five years. Thus, the Phillies were wrong to decline his option, correct?

Well, no. First, we have to come up with an estimate of Feliz' value if he plays 162 games. Then we have to discount that figure by the probability that he gets injured. Playing next season at age 35, and with this probability is increasing year-over-year at a faster rate than it ever has, it is likely that there is at least a 10% chance Feliz spends a month or more on the disabled list. There is also a non-zero chance he suffers a career ending injury, etc. 

So if our original estimate is that Feliz' value will be approximately $5 million in the event that he is healthy, it is almost certainly true that the Phillies made the right decision in declining his option. 

This conclusion is reinforced once we take the necessary step of evaluating the other options. The two most commonly discussed are Chone Figgins and Adrian Beltre.

Here's a typical discussion of Figgins, who is rather difficult to value:

So with all this information at our disposal, I ask – what’s the value of Chone Figgins? Since 2007, FanGraphs calculates that he has been worth a total of $50.9 million, with a high of $27.4 million in 2009 (while getting paid a paltry $5.78 million). But with Figgins finally eligible for free agency, would you pay Figgins $50 million over the next three years?

Figgins' FanGraphs values are even more all over the map:

Team WAR Value
2005 Angels 2.7 $9.20
2006 Angels 0 $0.10
2007 Angels 3.1 $12.50
2008 Angels 2.4 $11.00
2009 Angels 6.1 $27.40

And here's a graph:

Figgins_medium

Eyeballing that chart, you are likely willing to discount his outlier season of 2009, despite the fact that it is the most recent. It was fueled largely by a career high 16.3 fielding runs above average (by UZR), and perhaps you don't trust UZR's year to year fluctuations (motivated by good reasons).

So let's say you throw out his outlier 2006 and 2009 seasons, and expect him to be worth between $9 million and $12 million. We'll split the difference and say $10.5 million. Again, that is if he is healthy. He will play next season at age 32, so perhaps the risk there is lower than it is with Feliz.

But does the bigger fluctuation in value (think of a beta value) make Figgins worth less than his expected value? Possibly. If teams are looking for production certainty, they may be willing to pay a premium. However, we ought to wonder whether the fluctuations are related to something inherent about Figgins, or are simply a combination of luck and circumstances. 

In any event, if the Phillies are going to pay what is essentially par for the third baseman, perhaps they would like to guarantee a higher level of production than that which Feliz can be counted upon to provide.

The important thing is not to allow the ready availability of FanGraphs numbers, which are evaluated in hindsight, color your expected valuations. To do so is to fall prey to a common set of cognitive biases. 

And, of course, if they want to make the ballsy play, they'll go after the riskiest but also probably the best option: Adrian Beltre.

0 recs  |  Comment 11 comments  |  Add comment |

Story-email Email Printer Print

Comments

Display:

On UZR's purported unreliability

http://www.fangraphs.com/blogs/index.php/uzr-2008-to-2009/

So the lesson is, when there’s not a lot of UZR data on a player, there will be a lot of noise, but as the sample size increases, the data (at least from 2008 to 2009) actually becomes almost as highly correlated year to year as the stats that are considered to be the most reliable. (like wOBA)

by Crashburn Alley on Nov 9, 2009 12:40 PM EST reply actions   0 recs

Yeah, I agree with that assessment.

But I do think the park adjustments are poor.

by Tommy Bennett on Nov 9, 2009 12:41 PM EST up reply actions   0 recs

For example

I am not sure Feliz’ defensive decline from ‘06 to ’07 isn’t attributable mostly to differences in home park.

by Tommy Bennett on Nov 9, 2009 12:46 PM EST up reply actions   0 recs

Nagging injury?

Aaron King is still my homeboy... iffy mechanics and all

McFAQ for all you newcomers out there.

GET THAT VORP AND WHIP SH!T OUTTA HERE!!!

by baetown415 on Nov 9, 2009 3:55 PM EST up reply actions   0 recs

The answer is a bit murkier than that.

The issue is playing time. The average qualified starter gets 270 expected outs a season or so (what I presume is being referred to as UZR chances there). A qualified starter gets about 600 PAs, to contrast.

Also, remember that you have a smaller spread in UZR than you do in offensive metrics due to selective sampling (well, that’s not the ONLY reason, but it’s certainly a big reason and the one most relevant to this discussion). In other words, if we were to similarly break out wOBA relative to position we’d probably see the correlation incease as well, because the variance would be smaller.

UZR is unreliable. It is more unreliable than wOBA (which is, it should be noted, also somewhat unreliable). Defense also has a very different looking aging curve from offense, which I think some people forget. (Defense peaks basically at debut.)

And here’s a little brain teaser for everyone. When regressing UZR to the mean, what mean should we use? For instance, let’s say you have a -11 UZR/150 player at shortstop. Do you expect him to regress to the mean of all shortstops, and improve? Or to regress to the mean of all MLB players, and decline (or more likely, be moved off the position altogether?)

by cwyers on Nov 9, 2009 1:18 PM EST up reply actions   0 recs

It's an interesting question.

The easiest answer is, as you say, to regress to the position mean. It will appear to be the most “correct” – in other words, players that play in consecutive seasons will regress toward the positional mean.

Where this gets interesting is that you are, in fact, only looking at players able to stay at the position. Let’s consider, for a moment, our poor defensive shortstop again. If he is truly a decent enough shortstop, he will stay at the position (there are other factors, such as offense, to consider as well – I’m simplifying a bit). If he is not, he will likely either be moved to second or third, or simply not play in the majors the following season, depending on how good his bat is.

So if we want to answer the question of, “how good a defensive shortstop is this player?” then regressing to position seems correct.

But if we are trying to answer a question as posed in this article, are we only interested in projecting (say) projecting Figgins’ defense in that way?

by cwyers on Nov 9, 2009 2:33 PM EST up reply actions   0 recs

Is this a question that could be answered by the data?

I’d assume one regression does a better job at forecasting than the other.

My intuition tells me you should regress to the mean of all players, the reason being that you do the exact same thing with other statistics. You don’t regress wOBA on a positional basis, which is a measure of one’s offensive talent. Why do that with defensive talent? You’re basically regressing to a sub-group of players with a “similar” ability level.

by shawndgoldman on Nov 9, 2009 2:25 PM EST up reply actions   0 recs

You're comparing different types of statistics here.

UZR is more like a central moment in that it is a comparison of each value against some mean.
A shortstop with UZR = 0 is different from a first baseman with a UZR = 0. Putting the first baseman at short would result in a much lower UZR — the positional adjustments try to capture this aspect with a simple adjustment that says that a shortstop of UZR = 0 is more like a UZR of 5 when compared against the “average” player, while a 1B with a UZR = 0 is more like a UZR of -8 when compared against the “average” player.

wOBA is a general moment — it is not centralized, and therefore rather than being compared to zero, it is compared to a league average of ~.330 – so regression to the mean, in this case, is straight-forward. There’s no +5 runs given to a 1Bs offense over SS as each single, walk, or home run counts the same.

As such, I’d argue that UZR needs to be treated on a positional basis since the stat itself is dependent upon the performance of others at that position – or at worst regressed to the mean AFTER positional adjustments; while wOBA is calculated independent of position.

by Trickman on Nov 9, 2009 2:41 PM EST up reply actions   0 recs

I don't know if it's a question that can be definitively answered at all, at least not as posed.

The correct answer is probably “it depends on what question you are attempting to answer with the data.”

And yes, due to the selective sampling issue, one is always going to do better at predicting what is observed – in this case, regressing to position mean is always going to be more accurate than regressing to league mean, looking only at players who play the same position in consecutive seasons. But does being more accurate in that narrow sense make it more “correct?” Again, there is not one good answer for this.

A similar issue arrises when it comes to projecting players with little to no major league experience. All projection systems I am aware of regress translated stats to the MLB mean. And that is going to provide, in retrospect, the best projections of those players who played in MLB. But does it do a very good job of projecting what rookies are ready for MLB, and which aren’t?

by cwyers on Nov 9, 2009 2:47 PM EST up reply actions   0 recs

Here's where regressing to a supposedly "unbiased" scouting report would really help

MGL was talking about this last week. Then again, where are you going to find such a report and how are you going to convert it to a meaningful number?

by SFiercex4 on Nov 9, 2009 4:39 PM EST up reply actions   0 recs


User Tools

We use numbers and stuff.
Community Guidelines
Why be a member?
Start posting on Beyond the Box Score »

Join SB Nation and dive into communities focused on all your favorite teams.

FanPosts

Community blog posts and discussion.

Recommended FanPosts

Small
PZR-based Win Values 2001-2006
Small
The "30 parks on a budget" challenge

Recent FanPosts

Small
Determining Batted Ball Rates using Pitch Type and Location
Small
a new xBABIP calculator
Img587561916661595
Top 15 high school MLB draft prospects
Sunflower_small
World Series Simulation, Game #6
Small
JT20 Dynasty League
E52205a2_small
New Look
Sth70021_small
Exploring Hit f/x, Albeit Badly
Redcap_small
Ricky Nolasco: 4 WAR or 1 WAR?

+ New FanPost All FanPosts >

FanShots

Quick hits of video, photos, quotes, chats, links and lists that you find around the web.

Recent FanShots

Primer on BaseRuns
Cool Baseball Infographics
ESPN's Jerry Crasnick on defensive metrics
I’m also a follower, since Brian Bannister’s on our team, of sabermetric st...
Top Ten Baseball-Reference.com's Sponsorships
Primer on Linear Weights
JC Bradbury on "Hot Stove Myths"
Everyone Should Learn to Throw a Cutter
Criminals of WAR
Ten statisticians you should know about

+ New FanShot All FanShots >

BtB on Twitter

Main Feed: @BtBScore

Tommy B: @tommy_bennett
Sky: @BtB_Sky
Dan: @dturkenk
Harry: @harrypav
Jinaz: @jinazreds
Jack: @jh_moore
Erik: @Erik_Manning
Tommy R: @trancel
Justin: @justinbopp

Subscribe to BtB via Email

Enter your email address:

Delivered by FeedBurner

BtB Goes Social


Managers

Nando_small R.J. Anderson

Limes_125_small Sky Kalkman

E52205a2_small Tommy Bennett

Editors

Face_small Harry Pavlidis

Rawlings_baseball_bigger_small Dan Turkenkopf

770insig_small Jeff Zimmerman (TucsonRoyal)

Aviles_small Justin Bopp

Authors

Banny_small erik

Raysring1_small Tommy Rancel

Jinaz-reds-avatar_small JinAZ

Jmlogo_small Jack Moore

1753738656_110919ebe9_o_small vivaelpujols

1_small Graham

Baseball_small Mike Rogers

Redcap_small SFiercex4

Small Patrick Clark

Walter_album_small Walter Fulbright