What exactly do we mean by true talent level?
Justin's written about the AL and NL All-Star teams (NL Hitters, NL Pitchers, AL hitters, AL pitchers) recently, and used true talent levels to pick the best teams possible.
Is true talent level the same as a projected level of performance?
If we know a player is hurt (say, Grady Sizemore out for the season), does that affect his true talent level? If so, how? What about a smaller injury?
To what extent can we actually know a player's true talent level? We tend to use multiple years worth of data, but how much does 2007 really have to do with a player's 2010 level of production?
How different are hitters and pitchers? Do we want different types or amounts of information to estimate their true talent levels?
Can we use scouting information or pitch f/x information to hone in true talent level, going beyond conclusions made from more traditional performance metrics?
A question from Sky
almost 2 years ago
Sky Kalkman
13 comments
0 recs |
Email
PrintComments
True Talent levels
seem to me like a figment of our analytical imaginations. Taking past performance, knowledge of league norms and removing random variations has a good predictive quality and so we assume that results are also good estimations of true talent, but it isn’t really true. Players are always learning, always trying to improve and strength their game. Some of the biggest discrepancies in projections come from players who make a seemingly small change in their game and play at new level (we focus on the improvements alot, but it goes the other way too).
Ben Zobrist’s newly retooled 2009 swing was a big story for that reason. This year there is Mike Pelfrey and his splitter. Their talent allowed them to become the new, better player, but previously it was hidden by a flaw in technique. Just using past data cannot account for that progression and is thus more of an “accessible skills rating” than a “true talent level”. Unfortunately, that phrase is not very catchy and probably should not be used by anyone ever again.
- Matt Sullivan
A good friend of mine used to say, "This is a very simple game. You throw the ball, you catch the ball, you hit the ball. Sometimes you win, sometimes you lose, sometimes it rains." Think about that for a while. - Nuke LaLoosh
I pretty much agree with all that.
Do you think players have true talent levels different from our estimations that we call true talent levels? Or is it something we should find another name for because it doesn’t exist?
by Sky Kalkman on Jun 13, 2010 12:44 PM EDT up reply actions
Answers
A player playing through injury doesn’t affect his true talent level. If he was healthy, we would be able to play to his TTL. However, if a past injury affects future performance (i.e. arm injuries forcing a drop in pitcher velocity), then their TTL has changed. Do we think Sizemore’s injury will make him a worse player in the future?
Pitchers are hard to gauge, but Pitch f/x will be what we use. We don’t have the sophistication we need with Pitch f/x to measure a pitchers true talent, because we still can’t predict where they were trying to throw the pitch (vs where it ends up). We’ll get there, but we aren’t there yet.
Overall, TTL is closely related to projected performance. BUT, that projected performance needs to be context neutral.
Jesse wondered about Liriano's "true talent level" (my words) in another thread...
Given his 2010 line:
FIP: 2.09
K/9: 9.71
BB/9: 2.34
HR/9: 0.22
I think many people would agree that “he’s a different pitcher this year than in 2009”. But how do we know that? Are we looking at his results? I don’t think that fair. What I DO think is fair is looking at his scouting report/pitch f/x data — if he actually DOING things differently, then we can put more weight in his results. If he’d doing the same thing (same pitches, same velocity, etc.) then I’d be willing to put less weight on this season.
But I have no idea if that’s “legal”.
Good Example
Maybe TTL and a player’s “potential” are the same thing. Meaning, we think/know they have the ability to be very good, so wouldn’t that be their TTL?
by Zach Sanders on Jun 13, 2010 12:57 PM EDT up reply actions
I assume you don't mean "down the road" by "potential", though, right?
Something like “current potential”? Does that ignore downside, though?
Um yes?
http://www.hardballtimes.com/main/article/projecting-hanson/
This method would pick up on Liriano’s improved fastball velocity.
Agreed.
And that speaks to my response to Sky’s question. I really should have said “estimated” true talent level in that series. True Talent level is what projection systems try to estimate. It’s never actually going to be measurable, but as projection systems improve we’ll get better and better at estimating it. Nick’s approach above, once it’s been honed a bit (or maybe even now), should do much better at estimating true talent level simply because it includes more information.
-j
I write at:
Beyond the Boxscore | Red Reporter | Basement-Dwellers.com | Twitter: @jinazreds
By the way Justin, not that it means much of anything...
But Hanson has a 3.69 ERA this year. From my article:
After regressing each component of Hanson’s past performance to the mean of his most similar pitchers and plugging those number through the BaseRuns formula, I get an expected ERA of 3.67 going forward for Hanson.
Hehe ;)
by vivaelpujols on Jun 14, 2010 2:34 AM EDT up reply actions
That has to be unbelievably gratifying
Even if it’s pure luck! :)
-j
I write at:
Beyond the Boxscore | Red Reporter | Basement-Dwellers.com | Twitter: @jinazreds
I'm late to the party, but my stab at it
What exactly do we mean by true talent level?
True talent level is a player’s fundamental ability to perform. It’s what their rate stats would look like given an infinite number of plate appearances that happened instantaneously. It’s how good a player actually is.
Is true talent level the same as a projected level of performance?
Absolutely not. We often see people assume projected performance = true talent level for their purposes, but true talent level is not something we can measure or forecast.
If we know a player is hurt (say, Grady Sizemore out for the season), does that affect his true talent level?
Almost certainly, both in the interim (because hurt players can’t play) and in the long run (injuries often effect a players ability to perform in the future).
If so, how?
By changing the player’s body, which changes his ability to perform. The reality is true talent level isn’t something that would remain the same even in the absence of an injury. It not only changes year-to-year, but day-to-day, PA-to-PA, and even second-to-second. The human body is a lot more volatile than we might think.
What about a smaller injury?
Losing or gaining an ounce of body fat changes true talent level. Everything effects it, especially injuries, no matter how small.
To what extent can we actually know a player’s true talent level?
It isn’t something we can effectively measure or forecast (at this point, at least), so not at all. We can only estimate.
We tend to use multiple years worth of data, but how much does 2007 really have to do with a player’s 2010 level of production?
How different are hitters and pitchers? Do we want different types or amounts of information to estimate their true talent levels?
Pass. I’m not informed enough on the subject to give a good answer.
Can we use scouting information or pitch f/x information to hone in true talent level, going beyond conclusions made from more traditional performance metrics?
I don’t think we can scientifically integrate scouting information into our estimates of true talent level. What we can do, as always, is use scouting information as a sort of after-market correction for a projection. My guess is this is what the most saber-inclined teams do.
Beyond the Box Score / Capitol Avenue Club / shwitter: @CapitolAvenue
by PWHjort on Jun 14, 2010 12:02 AM EDT reply actions 3 recs
True talent level is an abstract idea, like PWHjort says, what we’d expect a guy to do in an infinite number of trials. Essentially, it’s the population mean. Seeing as how we’ll never have the full population to measure, we can never know it; all we have are samples to estimate it to the best of our ability. Projections are merely our best estimates as to what a players true talent is based on the samples of his performances we have.
We tend to use multiple years worth of data, but how much does 2007 really have to do with a player’s 2010 level of production?
How different are hitters and pitchers? Do we want different types or amounts of information to estimate their true talent levels?
Questions that need to be answered empirically. Essentially, the projections systems are our best guesses as to the answers to these questions based on historical precedents. I don’t know the details of the systems, so that’s all I can answer on the subject.
Can we use scouting information or pitch f/x information to hone in true talent level, going beyond conclusions made from more traditional performance metrics?
Absolutely. Especially something like pitch f/x, once a lot more research has been done on it, should provide some great information at the pitch level for us that simply looking at the end results won’t tell us. Pitch level information >>>>>> just looking at the end results. I’ve been waiting for pitch f/x stuff to come out for a while since I think a number of faulty conclusions were made from the end results (like pitchers don’t control BABIP, though everyone is beginning to figure that out now). The “end results” level is going to contain a lot of noise that hides the answers we’re looking for, so we really can’t be as confident in all our conclusions as people sometimes tend to think we can…especially when that conclusion is “we didn’t find anything”.
Scouting information can also be included. Don’t agree with PWHjort at all – you can include scouting information into a statistical model if you want, it’s not that difficult, and it may provide valuable information that makes the model a better estimate of “true talent”.
Comments For This Post Are Closed
Secondary Sidebar
User Tools
FanPosts
Community blog posts and discussion.
Recent FanPosts
Follow us on Twitter!
BTB: @BtBScore
Justin: @justinbopp
Satchel: @SatchelPrice
J-Doug: @RationalPastime
Josh: @garik16
Julian: @giantsnirvana
Bill: @BillPetti
Lucas: @DBITLefty
Jeff: @jeffwzimmerman
Lewis: @LewsOnFirst
