So how consistent is defense?

by RollingWave on Nov 12, 2008 1:23 AM EST

I'm looking at the recent PMR posts, and comparing them with PMRs from last year, and what really struck me was how inconsistent players seem to be from year to year.

There are even cases where someone goes from really bad to pretty good (Hanely) or vice vera (Coco Crisp) and some known long time attrocities (Mr. Jeter) were able to make remarkable recovery in his mid 30s. and while there seem to be some consistency, as a whole it just seem to flucate pretty wildly.

so while I agree that defense is the one area where some teams are really holding a edge over others, the question begs that how consistent are these results and what should we really make of it? while obviously a known good glove like Mark Teixiera would surely out perform the statue of Jason Giambi in any given year. on a more general bases, is it truely a good idea to base signings and trades heavily on defense?

Tweet Comment 12 comments | 0 recs |

Do you like this story?

Comments

This is a great question.

I think I remember something about one year of hitting data being as conclusive as two years of fielding data. But don’t hold me to that.

There are a few levels you can think of the data being inconsistent:

Players are actually inconsistent.
There’s a lot of noise, making more data necessary to pick out the real talent levels.
There’s inaccuracy in the stats, making more data necessary to weed out the error.

Those aren’t independent, either.

Beyond the Boxscore // Calling BJ Upton lazy is lazy.

by Sky Kalkman on Nov 12, 2008 2:24 PM EST reply actions

I'd heard three years of defensive data.

J.K.L.

by Aaron Campeau on Nov 12, 2008 7:48 PM EST up reply actions

Pizza Cutter has a great pair of articles on individual metrics point of becoming meaningful.

He uses an r^2 of .50 as the cutoff:

http://statspeak.net/2007/11/525600-minutes-how-do-you-measure-a-player-in-a-year.html

50 PA – swing percentage
100 PA – contact rate, response bias (both just missed at 50… the real number is probably around 70)
150 PA – K rate, line drive rate, pitches/PA
200 PA – BB rate, grounder rate, GB/FB ratio
250 PA – flyball rate
300 PA – HR rate, HR/FB
350 PA – sensitivity
400 PA – none
450 PA -none
500 PA – OBP, SLG, OPS, 1B rate, popup rate
550 PA – ISO
600 PA – none
650 PA – none

And for pitchers:

http://statspeak.net/2008/01/on-the-reliability-of-pitching-stats.html

Beyond the Boxscore // Calling BJ Upton lazy is lazy.

by Sky Kalkman on Nov 12, 2008 2:40 PM EST reply actions

How about some error bars?

It’s become extremely commonplace to use defensive stats such as Lewin’s +/- system to calculate the number of wins a player is worth in their valuation. But like you said, these can be quite variable. It’d be nice to have something rough like a 75% confidence interval with these types of stats.

by FTLOTC on Nov 12, 2008 5:31 PM EST reply actions

Generally, I prefer (when there is a large enough sample size) to see a range used.

So instead of saying “Player X is a +5 defender” you say “Player X is somewhere between a +0 and +10 defender.” When there’s enough data (three full seasons is my comfort level) I think you can make a fairly accurate assumption as to the defensive value of a player when you factor in injury history, aging curves, scouting impressions, etc.

When you’re dealing with young players with less than ample data, you’ve got to weigh the scouting reports to a greater extent. That’s not to say the numbers should be ignored, just that they should be seen for what they are; informative, but too small of a sample size to make any firm conclusions.

One of the goals of sabermetrics is the removal of as much subjectivity as is possible from baseball analysis. With that said, it is irresponsible (and counter-productive) to not realize that there are some things that aren’t quite as quantifiable as we’d like them to be; defensive analysis is certainly one of those areas that needs improvement.

I will say that I am comfortable enough with some of the advanced fielding metrics that a single season’s worth of data is enough to draw a reasonable conclusion about the value of a player’s contribution. I just think it’s important to know the difference between value in terms of what has already occurred and predictive value of analysis.

J.K.L.

by Aaron Campeau on Nov 12, 2008 8:10 PM EST reply actions 1 recs

Also, as far as the defensive value of catchers, I think it's fair to say that we know pretty much squat.

I think analysis of catcher defense is almost entirely subjective at this point.

J.K.L.

by Aaron Campeau on Nov 12, 2008 8:11 PM EST up reply actions

Agreed.

by R.J. Anderson on Nov 12, 2008 8:19 PM EST up reply actions

My guess is that

Well, I’m no stats wiz here, but from a general perspective, I think the biggest factor is a combination of….

A. Positioning, B. Luck.

it doesn’t exactly take stats to figure this out, it’s sort of a BABIP of defense problem, since most major leaguers aren’t going to drop a ball they can reach and things like throwing arm are a different subject altogether . then the problem is wether they can reach the ball.

obviously two aspect impact wether they can reach the ball.

1. their range

2. their positioning.

clearly, 1 is the real ability of a athelete that’s measurable, some guys cover more ground than others, and/or have better judgement /read that they react better. but the problem is that 2 is often just as important if not more in the outcome, it doesn’t take any stats to realize that even the most atheletic fielder ever won’t reach line drives hit far way from them, while even a statue would catch the same line drive if it was hit right at them.

My guess on the unpredictablity is simple, it’s because of positioning, we can look at the same player just to cut down the complexibility, if the same player and the same exact amount of balls hit to the same exact places, then where he stands obviously impact a ton, if he just happen to stand within reach of the ball a lot more often than normal (luck plays a major factor here, but maybe not complete factor) then he’s going to come out looking at lot better. if he just happen to have the poor luck (or some other reason) of standing in the wrong place at the wrong time more often than normal, than obviously he’s going to look bad.

I think this is the best reason to explain why almost all fielding stats seem to vary quiet a bit on a year to year bases for players.

the real question then is …

a. are there effective ways to position your players so they come out better? (I’m sure they have scouting report on those stuff, but often players seem to just position themself by their own desire as well)

can we or should we take out the positioning aspect out of the stats?

c. how much of positioning is a result of experience and/or coaches or scouting reports? and how much is on the players themself? do some players simply position themself better? do teams?

just my 0.02

by RollingWave on Nov 14, 2008 10:52 AM EST reply actions

a) absolutely – primary factor is batter, secondary is game situation (for example "doubles defense with the corner infielders tighter to the lines and outfielders deep) as well as factors like pitcher or playing conditions

b) not sure what you mean, but most stats don’t consider positioning

by ol Pete on Nov 19, 2008 12:51 AM EST up reply actions

positioning is important

but i think part of being a great defender is positioning yourself well. for example, I seem to remember Bill James talking about Jackie Robinson not being known for his defense, but having great fielding numbers at several different positions, which he attributed to Jackie’s intelligence and understanding of the game

"The NY Mets are my favorite squadron" --Apu Nahasapeemapetilon

by jessef on Nov 21, 2008 3:05 PM EST up reply actions