When should you pull the starter?

The game is on the line. Your starter is tiring. There's a man on base. Your relief ace is warming. The entire baseball world is watching you and people on Twitter are already armchair managing. Should you pull the starter or should you let them face the next batter? Most people will judge the outcome, but many of us will consider the process. You hardly ever get railroaded for a bad decision that succeeds, but if you make the right choice and it fails or if you make the wrong choice and you fail, you might be in trouble. What do you do?

Based on the responses to Ned Yost's decision to yank Shields and Matt Williams' choice to pull Jordan Zimmermann, you might think leaving the starter in the game is wise. Or you could observe Bob Melvin's decision to leave Jon Lester in too long as evidence of the other side.

So what factors should we consider when making a decision about whether to pull the starter or leave him in? Before we populate a list, we have to consider an important thing: All future decisions should be based on a projection about the future. That doesn't mean you have to run a simulation or a regression, but you always have to say "I think Pitcher X is this good against the next batter." The factors you believe influence that projection may be up for debate, but it's always about forecasting.

Let's think about some relevant factors.

Quality of Pitcher(s) Available

The first thing you should always consider is the talent level of the pitcher in the game and the talent level of the pitcher or pitchers waiting in the pen. If it's Clayton Kershaw in the game and Phil Coke in the bullpen, that's much different than Robbie Ray on the mound and Kenley Jansen in the bullpen. Certainly, you care about the talent level of who is available, and specifically, you care about their projected talent level given the context of the game.

Who's Batting?

This is another pretty obvious factor. If it's a weak-hitting lefty, you can probably get by more easily than if you're looking squarely at Miguel Cabrera. Not only do you need to consider the talent of the batter, but also the potential to exploit the platoon advantage. If you can put a LOOGY against Josh Hamilton, you might want to go ahead and do that. This also applies to who is coming up in the lineup and the likely pinch hitters that could arrive.

Fatigue

The effect is up for some debate, but generally speaking pitching tired means you pitch worse. It doesn't necessarily trump other factors, but a great pitcher at 115 pitches or on short rest won't pitch like they do at 45 pitches and full rest. Conversely, the reliever in question will pitch differently during his third consecutive day of work than he will after sufficient time off. There are a lot of intervening factors, but on balance, you expect fatigue to play some role.

The Stakes

I'm not a believer in the whole "I looked him in the eye and knew he could handle it" thing, but there is certainly a different calculation in a winner-take-all game compared to an average July contest. In July, you might not be able to rely on your pen as much, because there are less frequent off days and pitching them too often will hurt their effectiveness and increase their risk of injury. You have to plan for the game you're managing and the games you know will come in the future.

In an elimination game, you forget about the future. If the pitcher is most effective, they get the ball. It doesn't matter if you're risking their availability for their next outing, because that next outing is very highly discounted. There was no need to save Shields' arm in the WC games, but there was also no reason to save the arms of the bullpen. In July, you let Shields pitch deeper in that situation more often than not.

Times Through The Order

The Times Through the Order Penalty (TTOP) has gained popularity among analytic fans, but it hasn't really made it mainstream yet. It's simple, though. The more times you face the same batter in a game, the more likely they are to tag you for a big hit. This is true for basically every pitcher in the league and the biggest step comes from the second to third time, with the fourth time also being noticeable. Generally, starters do better than average the first time through the order, about average the second time through, and then get worse for the third and fourth time.

There are some conditional factors such as temperature, time of day, and such, but on balance, the more times you see a hitter, the more likely you are to fail as a pitcher. You can read more about TTOP here.

Putting it all together

Rob Carr/Getty

If you're a manager, what thought process should you take? All of these factors matter, and some matter more than others. The first thing you need to do is estimate the quality of your starting pitcher versus the batter you expect them to face. Then you need to estimate the quality of the reliever against the batter you expect them to face, and any ripple effect the decision may have on future batters, innings, or games.

Typically, a good reliever will be better than a good starter across a small number of batters. If you have Craig Kimbrel and you need three outs, there's basically no starter who is likely to perform better. It doesn't matter if they're "cruising" or the batters "aren't getting good swings". A lights-out reliever is better than a tired starter against a lineup the third and fourth time through the order.

So it all really comes down to how you project each pitcher. You should universally deduct value from a pitcher who is tired or facing a lineup multiple times or facing a bad platoon situation. But how good do you think these two pitchers really are otherwise for one batter or one inning?

That's where the debate really gets tricky, because Clayton Kershaw is better than all of the relievers in his bullpen. The problem is that he's not better than them in his 8th inning (or 4th time through the order) than they are in their first inning or first trip around the lineup. It's very hard for people to internalize that factor.

Would you rather have Zimmermann or Drew Storen? Of course you want Zimmermann, but that's not the actual question. The actual question is if you want a particular version of one pitcher or a particular version of another in a specific situation. It's not a matter of "trusting an ace," it's a matter of deciding how likely they are to throw ace-level pitches for the next batter or two.

Looking back, I liked both controversial quick hooks and didn't like Melvin staying with Lester for too long. A lot of people are saying you need to ride your aces longer in the postseason, but if you have even a reasonably good pen, the opposite is true. Get the six good innings from your ace and then unleash every weapon in your pen.

We lionize guys who can pitch deep into games and value that they want the ball with the game on the line, but that doesn't always mean it's the right move. Zimmermann is much better than C.J. Wilson and he's much better at 100 pitches than Wilson. But is he better than Storen at 0 pitches? Usually not.

...

Neil Weinberg is the Associate Managing Editor at Beyond The Box Score, the Site Educator at FanGraphs, and can also be found writing enthusiastically about the Detroit Tigers at New English D. Follow @NeilWeinberg44