Worth the Money?
Editor's note: Please welcome Steve to Beyond the Box Score. You can also find his writing about the Cardinals at Play a Hard Nine. -TBB
Inspired by Neyer's Monday piece on Matt Holliday, I thought now was an appropriate time to take a look at the developing LF market. The top 3 sluggers still available according to Rallys FA Tracker all play the position and have been linked to contracts of varying length/magnitude in recent days. With those rumors in mind I wanted to investigate how likely each player would be to produce surplus value on said contracts. In this particular case I turned to a simple simulation I developed.
Simulation Overview
I originally developed the simulation to look at one year roster construction, with the intent of gauging how likely certain team compositions would surpass various WAR totals; however, the same logic could be applied to looking at the length of a contract instead. I originally posted an article about the simulation at my other digs here. Give that a read if you want some of the details. The cliff notes version is that it's a simple Monte Carlo that takes wOBA and UZR/150 projections, generates a simulated wOBA and UZR/150 from a truncated normal distribution centered on the projections and given a user input SD, and then calculates RAA for offense and defense. After the fact I convert RAA to WAR based on simulated playing time numbers.
Inputs and Assumptions
The age old saying of garbage in garbage out applies to most (all?) simulations, and this one is no different. That being said, this section on the inputs affects the results section way more than the underlying simulation itself. I used CHONE projections for wOBA (calculated off of the 2008 figures published by Tango here) and my own for UZR/150 (available here). For offensive aging I used MGLs newly published aging study (available in its entirety here). As you'll see in the results I used two different aging curves for comparison sake, the traditional delta method applied to 1995-2008, and the delta method applied using JC's playing time restrictions (10 years in the league and 5000 career PA). For defensive aging I leveraged Jeff's previous work. I also ran excursions based on varying playing time/injury assumptions. Each player has a perfect health (PH) run with constituted 625 PA and 115 DGs, and they have an injury run (INJ) that has a 10% chance of losing half a season to injury for each simulation run. If you want to throw stones, now would be the appropriate time...
Results
All of the following charts are the WAR CDFs based on the respective simulation outputs. We'll go in order of increasing contract magnitude, which means leading off we have Johnny Damon, who is rumored to be looking for something in the neighborhood of 3/39M. All of the charts have WAR along the x-axis and probability of achieving at least that WAR on the the y-axis (the more interesting number in this case is 1-p however, as that will be equal to the probability of surplus value). [UPDATE as Justin points out I botched the words here hard core. It is actually the probability that the player will accumulate less than the corresponding WAR. Words on the CDF]
I only used one aging curve for Damon as out in the years we're dealing with the difference would be minimal. From the look of these charts 3/39 is a BAD deal (~1% chance of having surplus value) and even 3/30 is only a slightly better idea (~15-20% chance of having surplus value).
Batting second in our lineup of LFers is Jason Bay, rumored to be looking at a 4/60M deal from the Mets
Not a pretty picture, with only a ~2% chance of having surplus value. However, there are those out there that think that Bay's defensive number have been hurt by the Monster and/or aren't as bad as they seem. With that in mind, I replaced my projections (-9.5 UZR/150) with those from CHONE (-4) and reran the simulation. Here are the results.
While I wouldn't call it a good deal, it would be a little more palatable with a 15-20% chance of having surplus value. Now what happens if we slow the aging curve by using the JC-like aging curve with the better defense assumption (a best case combination).
This pushes Bay to a ~25-35% chance of having surplus value. Still not a good deal, but it's the best we'll get with this set of input parameters.
Now on to the biggest money deal, Matt Holliday. When I started this analysis I was working under the 8/128M deal that had been floated around (and referenced in the Neyer article). A smaller 5/80M would only be better for the signing team. First the standard delta method
This is a bit shocking to me. I wouldn't have thought that Holliday would have such a good chance to have surplus value (~50-80%), and the JC aging curve only makes it more likely (80-95%).
A lot of his value over the 8 years is tied to his defense, especially relative to the other players we saw, but there is probably enough disagreement about his defensive value to run another experiment. Also of note, the linear decrease I used to age defense never had him "fall off a cliff" over the eight years. With that in mind I altered the aging factor on defense to be less linear over the course of the deal, and reran using the standard delta method. Here's the chart
So we're now at 30-50% chance of surplus which is more in line with my gut (not that my gut is worth anything). Even with this more pessimistic (realistic?) prediction, he still offers a better chance to have surplus value than the other two options presented.
Conclusions
If I were a team in need of a LFer (and clearly my favorite team is) I would prefer to snag Matt Holliday at the newly reported 5/80M over either of the other two options. Bay becomes an interesting option only if you are inclined to disagree with UZR (or think Fenway affects UZR), and Damon just isn't very interesting to me at all.
After going through the process on these three guys, I think I like the methodology, but there could be some tweaks on the input side. I know I could do better with aging defense. I mentioned earlier that it was truncated normal, and in that arena I need to be very careful about the upper and lower limits (especially wOBA). Either way, do ya'll think that graphs like these are worthwhile to look at or is are there too many variables to mess with to get a clear picture? I'm all ears (which is an ironic statement considering how long winded my first post here was)
1 recs |
13 comments
|
Comments
I heart the idea of this
Absolutely love the concept. Struggling a bit with my understanding.
1. The plots. I’m still on my first cup of coffee here, but I’m not getting it. You say:
All of the charts have WAR along the x-axis and probability of achieving at least that WAR on the the y-axis (the more interesting number in this case is 1-p however, as that will be equal to the probability of their being surplus value).
So, reading Damon’s plot with INJ, I think it says:
~0% chance of at least 3 WAR
~15% chance of at least 4 WAR
~40% chance of at least 5 WAR
~65% chance of at least 6 WAR
~90% chance of at least 7 WAR
~95+% chance of at least 8 WAR
That makes no sense to me and almost has to be backwards. Is it “probability of achieving this or lower WAR” instead? That would make more sense. Also, if so, I’d tend to think that the probability of 0 WAR should be non-zero (catastrophic injury).
2. I missed the prior article, but looked through it and it looks solid. The expected variance around each player looked like it was a question mark, though, and that’s critical to your probability estimates. I was wondering, when you looked at the difference between actual and projected WAR, if those values were normally distributed? I’d expect some substantial right-skew, because I’d expect that there would be more people massively undershooting their projection (injuries, etc) than massively overshooting it. This would result, I think, in StDev not being a good measure of the variance. Not sure on the direction (still on first cup of coffee), but I think it would overestimate the variance. Which would give some justification for the reductions you attempted.
Maybe you can use something like a “median squared error” to assess the variance and help you ballpark where it should be a bit better? This seems like something that has to be resolved for the method to be reliable.
3. I think you should hold the x-axis constant on your plots to better demonstrate the differences in talent. Maybe that’s not possible to do and still show differences in the curves, though. Axis labels would also be nice. :)
-j
I write at:
Beyond the Boxscore | Red Reporter | Basement-Dwellers.com | Twitter: @jinazreds
yeah I messed up the wording there hard core
I had it written up better, but more long winded, and in trying to get it shorter I totally cut the wrong part. I’ll fix it. Thanks!!
by stevesommer05 on Dec 17, 2009 10:03 AM EST up reply actions
Got it now
And now I can say that I absolutely LOVE the plots. love love love. Would love them more with axis labels, but that’s like saying I’d love my 3-year old more if she wouldn’t have temper tantrums. :)
-j
I write at:
Beyond the Boxscore | Red Reporter | Basement-Dwellers.com | Twitter: @jinazreds
Yeah
Looking back at it, I’d concur on axis labels… net time right?
by stevesommer05 on Dec 17, 2009 10:16 AM EST up reply actions
RE #2
I was actually planning on going back and righting up the analysis I did to determine some of the inputs, just haven’t gotten around to it yet. It’s definitely on the to-do list.
by stevesommer05 on Dec 17, 2009 10:22 AM EST up reply actions
Goodness I'm a typo king this morning
writing not righting… ugh
by stevesommer05 on Dec 17, 2009 10:33 AM EST up reply actions
Awesome stuff Steve
One thing I would like to ask is about the full health projection of 625 PA and 115 DG. Just estimating off the top of my head, 625 PA for a cleanup hitter constituted 143 games played according to the NL’s rate last season. Given that DG is supposed to be based on an average number of chances at a position, wouldn’t it make sense to use a value for it that’s closer to an expected games played?
I’m not sure if that changes the projections/odds by a whole lot, but I think it does undersell a few of the better defenders/help a few of the worse ones. Perhaps Holliday is most affected due to the length of his expected deal.
Marlin Maniac, a Florida Marlins blog
Come attend Intro to Sabermetrics 101!
Check me out at Beyond the Box Score as well.
A full seasons of DGs
Isn’t necessarily equal to a full season of games I don’t believe. For example, last year Holliday played 156 games and had 139 DGs (since like you said it’s chances based). That being said my PH lines could probably stand to have a few more PA and DGs
by stevesommer05 on Dec 17, 2009 11:19 AM EST up reply actions
Agreed that DG doesn't necessarily equal games played
But because they’re based on an average number of chances, in general they aren’t going to be terribly far off. I think the projection could use a few more games at least; in my head, a player who plays 143 games at any position probably won’t get that few a number of DG’s.
Marlin Maniac, a Florida Marlins blog
Come attend Intro to Sabermetrics 101!
Check me out at Beyond the Box Score as well.
yeah, it definitely warrants a re-look
I need to re-examine a couple of other inputs too… I’ll look into coming up with some PA-DG conversions by position, that way I have something to point to instead of “uhhh it felt right”.
by stevesommer05 on Dec 17, 2009 2:31 PM EST up reply actions
Steve, awesome work. Glad to see you get the gig here at BtB — these are awesome dudes. Won’t be long before THT or Fangraphs comes callin’.
This article is fantastic. Some axis labels and it’d be even better.
Read my work at Tigers By The Numbers.
Welcome, Steve. Great work.
let me know if I can assist in any bigger, more visually-assaulting imagery.
See Data Differently.
beyondtheboxscore.com | Twitter: @ justinbopp

by 






































