Navigation: Jump to content areas:


Pro Quality. Fan Perspective.
Login-facebook
Around SBN: Blake Griffin Slam Dunks: NBA Jam Style

Why Jonah Lehrer's Criticism of Sabermetrics is so Disappointing

Let me say this up front: I like Jonah Lehrer. I am a fan of his writing and generally find him quite informative and thoughtful. This probably explains my visceral reaction to his latest, a dismissive column at Grantland that criticizes the use of analytics in various sports, including baseball.

As JD mentioned, it's a bad column and disappointing. I don't simply claim it to be bad because I disagree with his conclusions. I say it's bad because it is sloppy and Lehrer is a smarter guy than this.

The argument in short:

My worry is that sports teams are starting to suffer from a version of the horsepower mistake. Like a confused car shopper, they are seeking out the safety of math, trying to make extremely complicated personnel decisions by fixating on statistics. Instead of accepting the inherent mystery of athletic talent — or at least taking those intangibles into account — they are pretending that the numbers explain everything. And so we end up with teams that are like the worst kind of car. They look good on paper — so much horsepower! — but they fail to satisfy. The dashboard is ugly, the frame squeaks, and the front seats make our ass hurt.

First, there isn't a single team nor analyst that claims that sabermetrics capture all there is to know about athletic talent and performance. Teams use a variety of other metrics and method to assess player talent as well as likelihood to succeed.

Second, and more important, Lehrer's main argument shouldn't be that teams are assembling bad teams because of a narrow-minded focus on things they can quantify. The argument should be that teams that don't think deeply about what are the right metrics and how much variance they account for in player achievement will fail just as much as those teams that used to generally ignore analytical approaches to the game.

Data and statistics are not to blame for bad decisions--their misapplication is. Lehrer is trying to get at this point, but his misinformed broadside against sports analytics makes it real easy to miss.

Star-divide

Moreover, Lehrer's examples are just plain bad.

One example he uses is the NBA Champion Dallas Mavericks. He claims that because the Mavs did not match up statistically against each of their playoff opponents, yet prevailed, shows that a reliance on these statistics is flawed.

Lehrer is both right and wrong.

He's right in that relying solely on these statistics and not taking into account context would be a mistake. Again, I am waiting for someone to raise their hand and say they advocate this, particularly on a game by game or series by series basis.

He's wrong in that this example in no way proves or disproves the usefulness or accuracy of the underlying analytics. The playoffs are a handful of 7-game series. Anyone with a basic understanding of probability knows this is the last place you would look to prove or disprove the effectiveness of such measures given the role of randomness and luck.

Finally, Lehrer continually claims that a long list of intangibles largely determines the outcomes of games. He goes so far as to say that:

not everything that matters can be measured, and that success in sports (not to mention car shopping) is shaped by a long list of intangibles.

Now, he may very well be right, particularly if we are trying to make point predictions. But my first question is, if it can't be measured how do we know that it affects outcomes they way he claims? It's like someone who sees a UFO jumping to the conclusion that since we can't explain what it is it must be aliens. If it's unidentified you have no justification for making that leap. In Lehrer's case, neither does he.

Furthermore, sabermetrics--and any analytics whose subject is people and not atoms--is all about context and probabilities. When players don't perform as expected it could very well be for "intangible" reasons. This doesn't negate sabermetrics, not one bit. It goes hand in hand.

Like with any technology, sabermetrics and other analytical approaches won't by themselves lead to optimal decision making. Their is a mindset that must accompany them. That's a message worth delivering. Unfortunately, Lehrer is the wrong messenger. At least, he was on this day.

Comment 27 comments  |  8 recs  | 

Do you like this story?

Comments

Display:

The funny thing is

that in asserting that the Heat were statistically superior to the Mavs, he’s intrinsically assuming that basketball statistics function basically the same way as baseball statistics — and they don’t. As M. Haubs points out, there were some strong statistical indicators favoring the Mavs over the Heat in that series. (True, most people didn’t take the Mavs all that seriously, but as John Hollinger showed in some detail, that’s not because the indicators weren’t there.)

by The Ancient Mariner on Jun 28, 2011 2:29 PM EDT up reply actions   1 recs

edit:

*Would any of us ever dare suggest that the established aging curve DOESN’T take notable exceptions for the greatest players of all time?

Managing Editor at Beyond the Box Score and MLB Daily Dish. Follow me @justinbopp

by Justin Bopp on Jun 28, 2011 11:51 AM EDT up reply actions  

Great point. I feel like anti-sabermetric pieces often stoop to this method and pose conclusions that sabermetricians do not claim to make. If someone makes the effort to understand these stats and what they measure and choose not to buy in, I am okay with that. But to just dismiss with a small understanding of what advanced stats try to measure is just unprofessional and careless.

by JoshuaR on Jul 2, 2011 6:18 PM EDT up reply actions  

Side note:

What’s with lumping the fantasy guy/sabermetrics guy together? There’s a lot of crossover, but damn if we’re not talking about two very different audiences.

Managing Editor at Beyond the Box Score and MLB Daily Dish. Follow me @justinbopp

by Justin Bopp on Jun 28, 2011 11:36 AM EDT reply actions  

Yup.

I didn’t realize that fantasy leagues used any statistics beyond the ones that you find in any box score. Obviously if you like baseball numbers, you’ll probably like fantasy baseball, but the thought processes used for real baseball and fantasy baseball are pretty different.

Yes, my real name is actually Satchel.
I'm a columnist for Beyond the Box Score and a writer for MLB Daily Dish.
Oh, I'm on Twitter, too.

by Satchel Price on Jun 28, 2011 11:37 AM EDT up reply actions  

The thing is,

it’s not very fun to use sabermetrics in fantasy, or the ones I’ve run all kinda got boring when all the rate statistics regressed to the mean around July.

Managing Editor at Beyond the Box Score and MLB Daily Dish. Follow me @justinbopp

by Justin Bopp on Jun 28, 2011 11:39 AM EDT up reply actions  

I've never really gotten huge into fantasy

I like the BOAB league, because it doesn’t require constant engagement and I’d rather evaluate my picks by WAR than HR/RBI/R/BA/SB, but I’ve always found baseball interesting without the added competitive feel of running a fantasy team.

Fantasy definitely adds that level of uncertainty because stats like RBI, R and BA are so dependent on things that won’t always level out over the course of a season, so you can just get lucky and feel like you made a great pick.

I don’t know why there aren’t more WAR fantasy leagues popping up, although I suppose that maybe I’m just not hip to that beat.

Yes, my real name is actually Satchel.
I'm a columnist for Beyond the Box Score and a writer for MLB Daily Dish.
Oh, I'm on Twitter, too.

by Satchel Price on Jun 28, 2011 11:53 AM EDT up reply actions  

Maybe it's just me, but I would like to point out that

I’m still fond of counting stats. Doubles, Ks, HRs, etc.

Managing Editor at Beyond the Box Score and MLB Daily Dish. Follow me @justinbopp

by Justin Bopp on Jun 28, 2011 11:56 AM EDT up reply actions  

Yup, I still sort through those numbers all the time

Just because I prefer ISO to grade power doesn’t mean that I don’t appreciate a nice HR total or whatever. Counting stats certainly have their uses.

Yes, my real name is actually Satchel.
I'm a columnist for Beyond the Box Score and a writer for MLB Daily Dish.
Oh, I'm on Twitter, too.

by Satchel Price on Jun 28, 2011 11:57 AM EDT up reply actions  

Speaking of which,

you notice the leader is Bautista at 23? “On pace” (or w/e) for 46. Feels small, doesn’t it?

Managing Editor at Beyond the Box Score and MLB Daily Dish. Follow me @justinbopp

by Justin Bopp on Jun 28, 2011 11:59 AM EDT up reply actions  

Said this on the other post, but

The Mavs are apparently the most data-friendly team in the league. If this is really the only example he’s got, then he’s really not even standing on one leg.

by notoutofthe on Jun 28, 2011 11:58 AM EDT reply actions   1 recs

Yeah, the Mavs example felt like a really odd choice to make

Yes, my real name is actually Satchel.
I'm a columnist for Beyond the Box Score and a writer for MLB Daily Dish.
Oh, I'm on Twitter, too.

by Satchel Price on Jun 28, 2011 12:06 PM EDT up reply actions  

I believe Lehrer followed up on this in a Wired column.

Here.

A few people have suggested that I erred in choosing the Mavs as my primary example because they relied heavily on statistics during the season. But that was my point! Here is the most “forward” thinking team in the league, and yet when it came to their lineup in the last three games of the Finals they chose to start a player who, at least according to every conventional statistical analysis, didn’t look so hot. But Carlisle wisely realized that Barea had other things going for him

Meanwhile Marc Cuban said:

"Roland [Beech of 82games, a “stathed”] was a key part to all this. I give a lot of credit to Coach Carlisle for putting Roland on the bench and interfacing with him, and making sure we understood exactly what was going on. Knowing what lineups work, what the issues were in terms of play calls and training."

There’s a good chance Beech was actually the one who recommended putting in Barea. Lehrer basically ignores the fact that the team he is using in support of his argument is actually more statistic reliant than any other team in the NBA.

"You can spend minutes, hours, days weeks or even months overanalyzing a situation; trying to put the pieces together, justifying what would’ve, could’ve happened – or you can just leave the pieces on the floor and move the **** on."
-Tupac Amaru Shakur

by NetsMets4Life on Jun 29, 2011 2:57 PM EDT up reply actions  

I think the popularity of Moneyball partially created this false dichotomy

of “statheads” vs. traditional scouting – who’s right and who’s wrong. That’s not meant to be an indictment of the book. But, I think that’s the takeaway from a lot of folks who read it (and this seems to be reflected in the trailer for the film).

"It wasn’t that it was slippery or anything like that. It was just, dadgum, my hands and the balls were so wet..." - Tim Hudson

by KoKo the Monkey (T-Bone) on Jun 28, 2011 12:15 PM EDT reply actions   1 recs

in addition...

As Justin points out, there is some work/internet commenters that proliferates bad sabrmetrics that is devoid of critical thinking. I’d wager many writers/people couldn’t tell you the definition of the word or its purpose.

by JD Sussman on Jun 28, 2011 12:25 PM EDT up reply actions   1 recs

fun fact

Brian Greene has an Erdos number of 3 and a Bacon number of 2. (Unlike some contenders, his Bacon number is totally legit, as it comes a movie in which he had a speaking part and was not playing himself).

Not actually affiliated with whygavs.

by WHYG Zane Smith on Jun 29, 2011 9:05 PM EDT up reply actions  

The Mavericks, meanwhile, heavily relied on statistics on order to improve their team.

And now they are champions. Lehrer even mentions this. He basically takes a dump on his own argument. Repeatedly.

"You can spend minutes, hours, days weeks or even months overanalyzing a situation; trying to put the pieces together, justifying what would’ve, could’ve happened – or you can just leave the pieces on the floor and move the **** on."
-Tupac Amaru Shakur

by NetsMets4Life on Jun 29, 2011 2:46 PM EDT reply actions  

Comments For This Post Are Closed


User Tools

We use numbers and stuff.
Community Guidelines
Why be a member?

Follow us on Facebook!

Follow us on Twitter!

SaberGraphics

Yahoo_full_count

MLB Daily Dish

Get the latest MLB Trade Rumors, Transactions, and News at MLB Daily Dish!


Managing Editor:

Jbopp-kc_small Justin Bopp

Columnists:

Adam_small adarowski

Dme_small Satchel Price

Closeup4_small J-Doug

Carlosicon_small Julian Levine

Billy_and_daddy_4th_of_july_small Bill Petti

Featuring:

Dayton_small Jeff Zimmerman

12475953_small Jacob Peterson

Recent_pic_pg_small Patrick Gordon

Btbpro_small Dave Gershman

Me_small Bryan Grosnick

229331_10150183361996591_674441590_6760167_6637860_n3_small Lewie Pollis

Img_3830_small David Fung

30472_1481067225243_1190689185_1381415_997334_n_small Glenn DuPaul

1mnvxku7_small joshuaworn

Set_small MattFilippi18

Photo0011_small Nathaniel Stoltz