Beyond the Box Score: An SB Nation Community

Navigation: Jump to content areas:


Pro Quality. Fan Perspective.
Login-facebook
Around SBN: NFL Week One: Previews and Predictions for all 15 games

Worth the Money?

Editor's note: Please welcome Steve to Beyond the Box Score. You can also find his writing about the Cardinals at Play a Hard Nine. -TBB

Inspired by Neyer's Monday piece on Matt Holliday, I thought now was an appropriate time to take a look at the developing LF market.  The top 3 sluggers still available according to Rallys FA Tracker all play the position and have been linked to contracts of varying length/magnitude in recent days.  With those rumors in mind I wanted to investigate how likely each player would be to produce surplus value on said contracts.   In this particular case I turned to a simple simulation I developed.

Simulation Overview

I originally developed the simulation to look at one year roster construction, with the intent of gauging how likely certain team compositions would surpass various WAR totals; however, the same logic could be applied to looking at the length of a contract instead.  I originally posted an article about the simulation at my other digs here.  Give that a read if you want some of the details.  The cliff notes version is that it's a simple Monte Carlo that takes wOBA and UZR/150 projections, generates a simulated wOBA and UZR/150 from a truncated normal distribution centered on the projections and given a user input SD, and then calculates RAA for offense and defense.  After the fact I convert RAA to WAR based on simulated playing time numbers.

Star-divide

 

Inputs and Assumptions

The age old saying of garbage in garbage out applies to most (all?) simulations, and this one is no different.  That being said, this section on the inputs affects the results section way more than the underlying simulation itself.  I used CHONE projections for wOBA (calculated off of the 2008 figures published by Tango here) and my own for UZR/150 (available here).  For offensive aging I used MGLs newly published aging study (available in its entirety here).  As you'll see in the results I used two different aging curves for comparison sake, the traditional delta method applied to 1995-2008, and the delta method applied using JC's playing time restrictions (10 years in the league and 5000 career PA).  For defensive aging I leveraged Jeff's previous work.  I also ran excursions based on varying playing time/injury assumptions.  Each player has a perfect health (PH) run with constituted 625 PA and 115 DGs, and they have an injury run (INJ) that has a 10% chance of losing half a season to injury for each simulation run.  If you want to throw stones, now would be the appropriate time...

Results

All of the following charts are the WAR CDFs based on the respective simulation outputs.  We'll go in order of increasing contract magnitude, which means leading off we have Johnny Damon, who is rumored to be looking for something in the neighborhood of 3/39M.  All of the charts have WAR along the x-axis and probability of achieving at least that WAR on the the y-axis (the more interesting number in this case is 1-p however, as that will be equal to the probability of surplus value). [UPDATE as Justin points out I botched the words here hard core.  It is actually the probability that the player will accumulate less than the corresponding WAR.  Words on the CDF]

Damonjc_medium

via img32.imageshack.us

 

I only used one aging curve for Damon as out in the years we're dealing with the difference would be minimal.  From the look of these charts 3/39 is a BAD deal (~1% chance of having surplus value) and even 3/30 is only a slightly better idea (~15-20% chance of having surplus value).

 Batting second in our lineup of LFers is Jason Bay, rumored to be looking at a 4/60M deal from the Mets

Baydeltabad_medium

via img682.imageshack.us

 

Not a pretty picture, with only a ~2% chance of having surplus value.  However, there are those out there that think that Bay's defensive number have been hurt by the Monster and/or aren't as bad as they seem.  With that in mind, I replaced my projections (-9.5 UZR/150) with those from CHONE (-4) and reran the simulation.  Here are the results.

Baydeltagood_medium

via img706.imageshack.us


While I wouldn't call it a good deal, it would be a little more palatable with a 15-20% chance of having surplus value.  Now what happens if we slow the aging curve by using the JC-like aging curve with the better defense assumption (a best case combination).

Bayjc_medium

via img706.imageshack.us

 

This pushes Bay to a ~25-35% chance of having surplus value.  Still not a good deal, but it's the best we'll get with this set of input parameters.

Now on to the biggest money deal, Matt Holliday.  When I started this analysis I was working under the 8/128M deal that had been floated around (and referenced in the Neyer article).  A smaller 5/80M would only be better for the signing team.  First the standard delta method

Hollidaydelta_medium

via img24.imageshack.us

 

This is a bit shocking to me.  I wouldn't have thought that Holliday would have such a good chance to have surplus value (~50-80%), and the JC aging curve only makes it more likely (80-95%). 

A lot of his value over the 8 years is tied to his defense, especially relative to the other players we saw, but there is probably enough disagreement about his defensive value to run another experiment.  Also of note, the linear decrease I used to age defense never had him "fall off a cliff" over the eight years.  With that in mind I altered the aging factor on defense to be less linear over the course of the deal, and reran using the standard delta method.  Here's the chart

Hollidayd_medium

via img24.imageshack.us

 

So we're now at 30-50% chance of surplus which is more in line with my gut (not that my gut is worth anything).  Even with this more pessimistic (realistic?) prediction, he still offers a better chance to have surplus value than the other two options presented.

Conclusions

If I were a team in need of a LFer (and clearly my favorite team is) I would prefer to snag Matt Holliday at the newly reported 5/80M over either of the other two options.  Bay becomes an interesting option only if you are inclined to disagree with UZR (or think Fenway affects UZR), and Damon just isn't very interesting to me at all. 

After going through the process on these three guys, I think I like the methodology, but there could be some tweaks on the input side.  I know I could do better with aging defense.  I mentioned earlier that it was truncated normal, and in that arena I need to be very careful about the upper and lower limits (especially wOBA).  Either way, do ya'll think that graphs like these are worthwhile to look at or is are there too many variables to mess with to get a clear picture?  I'm all ears (which is an ironic statement considering how long winded my first post here was)

1 recs  |  Comment 13 comments |

Story-email Email Printer Print

More from Beyond the Box Score

Break out the Bats

Jun 2010 by PWHjort - 5 comments

Comments

Display:

I heart the idea of this

Absolutely love the concept. Struggling a bit with my understanding.

1. The plots. I’m still on my first cup of coffee here, but I’m not getting it. You say:

All of the charts have WAR along the x-axis and probability of achieving at least that WAR on the the y-axis (the more interesting number in this case is 1-p however, as that will be equal to the probability of their being surplus value).

So, reading Damon’s plot with INJ, I think it says:
~0% chance of at least 3 WAR
~15% chance of at least 4 WAR
~40% chance of at least 5 WAR
~65% chance of at least 6 WAR
~90% chance of at least 7 WAR
~95+% chance of at least 8 WAR

That makes no sense to me and almost has to be backwards. Is it “probability of achieving this or lower WAR” instead? That would make more sense. Also, if so, I’d tend to think that the probability of 0 WAR should be non-zero (catastrophic injury).

2. I missed the prior article, but looked through it and it looks solid. The expected variance around each player looked like it was a question mark, though, and that’s critical to your probability estimates. I was wondering, when you looked at the difference between actual and projected WAR, if those values were normally distributed? I’d expect some substantial right-skew, because I’d expect that there would be more people massively undershooting their projection (injuries, etc) than massively overshooting it. This would result, I think, in StDev not being a good measure of the variance. Not sure on the direction (still on first cup of coffee), but I think it would overestimate the variance. Which would give some justification for the reductions you attempted.

Maybe you can use something like a “median squared error” to assess the variance and help you ballpark where it should be a bit better? This seems like something that has to be resolved for the method to be reliable.

3. I think you should hold the x-axis constant on your plots to better demonstrate the differences in talent. Maybe that’s not possible to do and still show differences in the curves, though. Axis labels would also be nice. :)
-j

by JinAZ on Dec 17, 2009 9:53 AM EST reply actions  

yeah I messed up the wording there hard core

I had it written up better, but more long winded, and in trying to get it shorter I totally cut the wrong part. I’ll fix it. Thanks!!

by stevesommer05 on Dec 17, 2009 10:03 AM EST up reply actions  

Got it now

And now I can say that I absolutely LOVE the plots. love love love. Would love them more with axis labels, but that’s like saying I’d love my 3-year old more if she wouldn’t have temper tantrums. :)
-j

by JinAZ on Dec 17, 2009 10:09 AM EST up reply actions  

Yeah

Looking back at it, I’d concur on axis labels… net time right?

by stevesommer05 on Dec 17, 2009 10:16 AM EST up reply actions  

RE #2

I was actually planning on going back and righting up the analysis I did to determine some of the inputs, just haven’t gotten around to it yet. It’s definitely on the to-do list.

by stevesommer05 on Dec 17, 2009 10:22 AM EST up reply actions  

Steve

This is fantastic. Welcome!

by Tommy Bennett on Dec 17, 2009 10:07 AM EST reply actions  

Awesome stuff Steve

One thing I would like to ask is about the full health projection of 625 PA and 115 DG. Just estimating off the top of my head, 625 PA for a cleanup hitter constituted 143 games played according to the NL’s rate last season. Given that DG is supposed to be based on an average number of chances at a position, wouldn’t it make sense to use a value for it that’s closer to an expected games played?

I’m not sure if that changes the projections/odds by a whole lot, but I think it does undersell a few of the better defenders/help a few of the worse ones. Perhaps Holliday is most affected due to the length of his expected deal.

by SFiercex4 on Dec 17, 2009 11:12 AM EST reply actions  

A full seasons of DGs

Isn’t necessarily equal to a full season of games I don’t believe. For example, last year Holliday played 156 games and had 139 DGs (since like you said it’s chances based). That being said my PH lines could probably stand to have a few more PA and DGs

by stevesommer05 on Dec 17, 2009 11:19 AM EST up reply actions  

Agreed that DG doesn't necessarily equal games played

But because they’re based on an average number of chances, in general they aren’t going to be terribly far off. I think the projection could use a few more games at least; in my head, a player who plays 143 games at any position probably won’t get that few a number of DG’s.

by SFiercex4 on Dec 17, 2009 2:24 PM EST up reply actions  

yeah, it definitely warrants a re-look

I need to re-examine a couple of other inputs too… I’ll look into coming up with some PA-DG conversions by position, that way I have something to point to instead of “uhhh it felt right”.

by stevesommer05 on Dec 17, 2009 2:31 PM EST up reply actions  

Steve, awesome work. Glad to see you get the gig here at BtB — these are awesome dudes. Won’t be long before THT or Fangraphs comes callin’.

This article is fantastic. Some axis labels and it’d be even better.

Read my work at Tigers By The Numbers.

by Mike Rogers on Dec 17, 2009 3:19 PM EST reply actions  

Welcome, Steve. Great work.

let me know if I can assist in any bigger, more visually-assaulting imagery.

by Justin Bopp on Dec 17, 2009 11:48 PM EST reply actions  

Comments For This Post Are Closed


User Tools

We use numbers and stuff.
Community Guidelines
Why be a member?
Start posting on Beyond the Box Score »

Join SB Nation and dive into communities focused on all your favorite teams.

Connect_with_facebook

FanPosts

Community blog posts and discussion.

Recent FanPosts

Small
Real Dirty Mets Looking for a saberminded author
Baseball_small
WAR By Decade: 1871-1879
Prosser_small
Cliff Lee: No longer invincible
Paige_small
Kelly Johnson Cleared Waivers; I Think That's Weird
Jeter_06_world_series_small
Top 10 players to start a franchise with revised.
Ballgame_2006_vs_texas_revised_small
The Myth of the Spoiler Returns
Small
Denard Span's Strikezone
Small
Matusz: Danks 2.0
Paige_small
I Think I Offended Juan Pierre
Leopold_butter_scotch_southpark_small
HOF/PED Quandry

+ New FanPost All FanPosts >

Sign up for the BtB Newsletter!

FanShots

Quick hits of video, photos, quotes, chats, links and lists that you find around the web.

Recent FanShots

Of course, historical trends do not always hold, as has been the case for Secret Sauce since ~2002. Take a look at the success of the Secret Sauce favorites in head-to-head competition since the playoffs expanded to eight teams in 1995.

Continue reading Weak Sauce? Secret Sauce's Predictive Capacity Wanes in Recent Years
2010 Fans Scouting Report
Plate discipline trends
What's Wrong With Mike Pelfrey?
Lightest Players in History (min 1000 PA or 500 IP)
Statistical Head Scratchers: The Sacrifice Fly
Adam Wainwrights Curve
Jose Batista Facts
A PitchFX look at how R.A. Dickey is able to change speeds with his knuckleball to be so effective
Out Rate: a simple new upgrade on OBP

+ New FanShot All FanShots >

BtB on Facebook

BtB on Twitter

RSS Feed: @BtBScore

Sky: @BtB_Sky

Jeff: @jeffwzimmerman
Steve: @steve_sommer
Dan: @dturkenk
Harry: @harrypav
Jinaz: @jinazreds
Jack: @jh_moore
Tommy R: @trancel
Justin: @justinbopp
Satchel: @SatchelPrice
Adam: @baseballtwit
Larry: @wezen_ball
Peter: @CapitolAvenue
Paul: @TheDiaTribe
Daniel: @CamdenCrazies
Matt: @devil_fingers

SBNation.com Recent Stories

SEATTLE - JULY 10:  Starting pitcher Felix Hernandez #34 of the Seattle Mariners celebrates after defeating the New York Yankees 4-1 at Safeco Field on July 10 2010 in Seattle Washington. (Photo by Otto Greule Jr/Getty Images)

MLB Power Rankings: On The Challenge Of Identifying A League's Best Pitcher

ANAHEIM CA - SEPTEMBER 08:  Jeff Mathis #5 of the Los Angeles Angels of Anaheim is mobbed by teammates after hitting a  walk off sacrifice fly to score Torri Hunter form third base against the Cleveland Indians in the 16th inning on September 8 2010 at Angel Stadium in Anaheim California.   The Angels won 4-3 in 16 innings.  (Photo by Stephen Dunn/Getty Images)

Mathis' 16th-Inning Sac Fly Lifts Angels Over Indians 4-3

Philadelphia Phillies' Jimmy Rollins, left, slides into home to score past the tag of Florida Marlins catcher Brad Davis on a single by Carlos Ruiz in the third inning of a baseball game, Wednesday, Sept. 8, 2010, in Philadelphia. (AP Photo/Matt Slocum) +2 updates

Phils Top Marlins 10-6, Jimmy Rollins Leaves Game With 'Hamstring Tightness'

More from SBNation.com >


Managers

Limes_125_small Sky Kalkman

Wbc_029_small Jeff Sullivan

Editors

Rawlings_baseball_bigger_small Dan Turkenkopf

Dayton_small Jeff Zimmerman

Aviles_small Justin Bopp

Paige_small Satchel Price

Authors

Jinaz-reds-avatar_small JinAZ

Face_small Harry Pavlidis

Newavatar_small Matt Klaassen

Wezenball-logo_small lar

Big_pun--300x300_small Tommy Rancel

Adam_small adarowski

Redcap_small SFiercex4

St_louis_cardinals_ce1141_003263_small stevesommer05

Small garik16

Julio_teheran_2_small PWHjort

Cclogo_small Daniel Moroz

Closeup4_small J-Doug

Nick_cage_small The DiaTriber