Smoothing out the quality start boundary

During the AL Wild Card game Tuesday night, the TBS crew used quality starts to illustrate Jon Lester's worth to the Athletics down the stretch. In his 32 regular-season starts this season, Lester has thrown 27 quality starts, including 19 consecutive dating back to June 12.

Of course, the definition of a quality start leaves room for criticism: As Ron Darling pointed out on the broadcast, a pitcher only has to complete six innings and allow no more than three earned runs to be credited with a quality start. A pitcher that allows three earned runs over six innings every start would end up with an ERA of 4.50, well above average in the current run environment*.

* - And all that is ignoring unearned runs, which don't affect a pitcher's ERA (or quality start determination), but are still runs. Consider Lester's August 17th start against the Braves: 6 IP, 4 runs, 3 earned runs, 1 quality start, 1 loss.

This prompted an interesting discussion on Tom Tango's website. Tango wrote:

It goes back to the ​problem that Bill James noted thirty years ago: the term QUALITY. It’s a word that doesn't lend itself to being defined however we want.... Had it been called a (Walter) Johnson game, then no one says a single word about it.

Absent a renaming, there are other ways to get around this problem. Commenters on that post proposed tying the quality start definition to game score, among other things. My suggestion was, since starts on the QS boundary aren't really quality, to redraw the boundary. I created a poll presenting a number of combinations of innings pitched and earned runs, and asked participants whether they considered each combination "quality" or "not quality."

To this point, eight readers have participated. The chart below shows the percentage of participants who considered each combination "quality," with green checkmarks denoting percentages of at least 50%. It's probably not too surprising that the majority of respondents didn't consider a 6 IP, 3 ER start to be quality, but it's a little surprising that they had no problem calling a 5 IP, 2 ER start (and even a 4 IP, 0 ER start) quality.

The original poll only included whole numbers of innings, because including fractional IP values would have made the poll far too long. But by using linear interpolation, we can estimate percentages for the missing IP/ER combinations. Below, we've including a 3-D surface plot showing the interpolated results.

Now that we have this surface, we need to draw a new boundary. I chose a simply majority (50 percent), but if you preferred, you could require a two-thirds or three-quarters majority. The figure below includes the original quality start area (in red), and the new "people's" quality start definition in blue. The new definition is more forgiving for starters who pitch deeper into games, and also more forgiving for those who don't go as far but keep runs off the board.

There's no sense coming up with a new statistic if we can't use it to measure pitchers. First, the gainers: These starters gained the most QS under the new definition from 2010 to 2013. There are a few bullpen-saving efforts in this set, but for the most part, the extra QS are all shorter starts with fewer runs allowed.

Most starters gain at least one QS under this new definition, but a few suffer by comparison. These pitchers lost the most QS, throwing the largest number of 6 IP/3 ER games.

This statistic is not very useful: The formula makes it more complex than the basic QS definition, but the binary classification makes it less useful than statistics like game score. On the other hand, the smoother boundary would mean fewer arguments about whether a given quality start is really good enough.

. . .

All statistics courtesy of Retrosheet and Baseball-Reference. Special thanks to the readers at Tom Tango's blog for participating in the poll.

Bryan Cole is a featured writer for Beyond the Box Score who enjoys watching the Royals' running game. You can follow him on Twitter at @Doctor_Bryan.