So You Want to Have a Good Bullpen

This time of year, I tend to get stuck staring at projections. I think it’s a mechanism I developed over time as an alternative to making too much of spring-training statistics. I like to look at the projected standings to give myself a constant reminder of how the whole baseball landscape shakes out, but I also like to look at historical projected standings to give myself a similarly constant reminder of how baseball can surprise. Every March, I go back through my projections spreadsheet to remember supposedly good teams that underachieved. And even more fun than that, I browse to remember supposedly bad teams that overachieved.

When I do that, I’m reminded of the unbelievable 2012 Baltimore Orioles. They might be the most improbable recent success story — that team won 93 games, after being projected to win a woeful 70. And the key to the club was that they had the league’s most effective bullpen. This was before the emergence of bullpen monster Zach Britton. And in a way, those Orioles are representative; many of the greatest recent overachievers have been powered by excellent bullpens. Between 2007 and 2008, for example, the (Devil) Rays’ bullpen went from worst to first. A strong bullpen can squeeze wins out of thin air. There’s nothing quite like one, when you’re looking for any kind of separator.

I don’t need to tell you a good bullpen is important. Every fan *knows* a good bullpen is important. The question is, how do you know you have a good bullpen? It’s not as easy as you’d think. Or, it’s exactly as hard as you’d think.

After the fact, I like to play with Win Probability Added, or WPA. It’s not supposed to be a predictive statistic, but, especially with bullpens, it can tell the right story. A bullpen that held onto its leads will have a high WPA. A bullpen with a few too many meltdowns will have a low WPA. Since you already know how this works, here’s a plot — last year’s 30 teams, in descending order of WPA from relievers.

By this measure, the 2017 Red Sox had baseball’s best bullpen. That wasn’t a total surprise — the group had been projected to be a little above average. But, for one thing, there’s a difference between being 12th and being first. And, for another, the Sox bullpen was projected in 12th while relying in part on Robbie Ross Jr., Carson Smith, and Tyler Thornburg. Those pitchers threw a combined 15.2 innings. So, the Red Sox overachieved. An even more fun example might be the Diamondbacks. They finished in seventh in WPA. They had been projected in dead last. In March, no one appeared to have a worse bullpen. Then it turned into a strength.

The Royals bullpen was projected in 28th, and it finished in tenth. I know that the projections were in WAR while we’re talking here about finishes in WPA, but there ought to be a close relationship between the two. Also, this works in the other direction. The last-place A’s were projected to have the 13th-place bullpen. The 28th-place Rangers were projected to have the 14th-place bullpen. The 27th-place Mets were projected to have the ninth-place bullpen. The 25th-place Mariners were projected to have the eighth-place bullpen. Every year, obviously, there are overachievers and there are underachievers. It’s just that bullpens can be the hardest to see coming.

Let’s play with something related. We can leave projections aside, for now. We’ll focus instead on year-to-year performance. I’ve decided to examine the past decade, beginning with the 2007-2008 seasons. Here is year-to-year team batting WPA.

WPA is known to be a noisy statistic, because it’s so dependent on leverage. One shouldn’t expect year-to-year relationships to be extremely strong. Here, we see an r-squared of 0.14. The relationship is positive, but arguably weak. Here’s the same kind of plot, now, but for starting rotations.

Same deal. We’ve got an r-squared of 0.14. Nearly identical-looking line of best fit. At last, the same plot, but for bullpens.

It’s not that there’s no relationship. Ever so slightly, you figure a strong bullpen will be fine again the following year, and a weak bullpen will be problematic again the following year. But with an r-squared of 0.04, you see there’s very little performance carryover. Compared to the previous two lines of best fit, the line you see here is 44% flatter. This plot is close to being random. It’s not all the way there, but bullpens simply don’t make it easy. Strange things can make or break a whole corps.

To a certain extent, you should expect this. Bullpens don’t throw that many innings, compared to rotations, and there can be a pretty high rate of personnel turnover. And because relieving is far easier than starting, some reliever conversions can suddenly hit the ground sprinting. In that way, Archie Bradley partially explains the success of the 2017 Diamondbacks. No one quite knew he’d adapt to relieving as well as he did. Overall, though, there’s just this strong whiff of unpredictability. Look at some of the bigger contracts given to guys who relieved in 2017. Wade Davis, Bryan Shaw, Addison Reed — they’ve had multiple years of effectiveness. But, Brandon Morrow got $21 million. Mike Minor got $28 million. Tommy Hunter got $18 million, and Anthony Swarzak got $14 million. Yusmeiro Petit got $10 million. Where were those guys a year ago? They all became phenomenally important out of nowhere. And so certain bullpens were made.

The fact that those out-of-nowhere relievers got big guarantees demonstrates that teams still do find individual performances predictive. As they should. In the big picture, effective pitchers should be expected to remain effective pitchers. But bullpens as units are fragile, and some good stat lines might be mirages. One injury might knock everything out of order. And on the other side of things, someone in the minors might take a leap forward, or some starter might suddenly feel at home in the eighth inning. Right now, the Yankees have the best-projected bullpen, and the Tigers have the worst-projected bullpen. And, honestly, the Yankees’ bullpen might be so good and so deep it’s almost immune to standard reliever volatility. But there’s a lot of wiggle room here. The Yankees, the Astros, the Cubs, the Indians — one of those bullpens could become trouble. And what if the Tigers get something big from Joe Jimenez? What if Warwick Saupold figures out a new grip? What if Shane Greene cuts his walk rate in half? The difference between good and bad in the bullpen is slim. Reliever life is a tightrope.

You’ve probably understood on some level for a while that bullpens are largely unpredictable. Teams understand that, too. At the same time, bullpens now are perhaps more important than they’ve ever been. What an encouraging thought for teams on the rise. What uncomfortable circumstances, for teams at the top.