The Extra Value of Having an Elite Bullpen by Jeff Sullivan November 20, 2015 Not too long ago, I tried to very simply figure out if there’s any extra value in having an elite reliever. I found some evidence of something, although it could stand to be confirmed, since in retrospect I don’t totally love the method I used. But generally speaking, it makes sense — a team could leverage that reliever particularly well, and bullpens are an area we have some trouble evaluating statistically. All that did was look at the potential impact of one guy. It was interesting to me, but there are other ways to try to break this stuff down. So for this post, I’m moving from the impact of an elite reliever to the impact of an elite bullpen. Why focus on one late-inning pitcher, when every team needs a few? Can we see any hidden extra value in having an overall strong group of relievers? Before, what I did was compare team wins to expected team wins, based on WAR. As much as WAR is imperfect, it’s great in the aggregate, and it tracks team wins very closely. I’m choosing to do that again, putting everything over a common 162 games to even out any little discrepancies. I went back 30 years, covering the window from 1986 through 2015. This gave me 862 individual team-seasons, and for each I calculated total WAR per 162, and wins per 162. Then, for each, I also calculated reliever WAR per 162, and that’s what I ultimately sorted by. For this first table, I separated out the 30 best bullpens, and the 30 worst bullpens, by WAR. Shown are all the relevant averages. As usual, it’ll make more sense after you look the table over. I’m not doing real well right now with words. Wins vs. Expected Wins, 1986 – 2015, Grouped by RP Strength Group W/162 expW/162 Difference WAR/162 RP WAR/162 RP Clutch/162 Top 30 90.0 88.5 1.5 40.9 7.2 1.8 Middle 81.1 81.0 0.0 33.4 2.5 0.2 Bottom 30 70.2 72.6 -2.5 25.0 -2.2 -0.5 The teams, of course, are differently good or bad, by group. The group of teams with the best bullpens is also just the best group overall; the group of teams with the worst bullpens is just the worst group overall. That’s not uninteresting, but it’s not the interesting thing here. Look at the first group. On average, the teams with the strongest bullpens won 90 games. But just based on WAR, they’d be expected to have won 88.5 games, so you get an average of an extra 1.5 “hidden” wins. The second group, of hundreds of teams, is right where you’d think. The last group shows up with 2.5 missing wins. Samples of 30 allow for there to be noise, but it looks like there could be something. This next table is the same as the above table, only instead of splitting away the best and worst 30, I’ve split away the best and worst 60. So, all the same stuff: Wins vs. Expected Wins, 1986 – 2015, Grouped by RP Strength Group W/162 expW/162 Difference WAR/162 RP WAR/162 RP Clutch/162 Top 60 89.6 88.6 1.0 40.9 6.5 1.3 Middle 81.2 81.2 0.0 33.5 2.5 0.2 Bottom 60 70.1 71.6 -1.5 23.9 -1.6 -0.4 Now the first group shows one extra “hidden” win, while the last group pulls into a more reasonable 1.5 “missing” wins. The middle group, again, is right where you’d think, with a difference of 0.0. Those middle-group teams had an average bullpen WAR of 2.5. The elite-group teams had an average bullpen WAR of 6.5. For reference, last year the Orioles just led baseball at 6.4, while the Astros and Yankees came in at 5.3. The year before, the Yankees were tops at 5.5. The 2003 Dodgers were amazing, at 9.3, thanks to Eric Gagne, Guillermo Mota, and Paul Quantrill. That’s the best bullpen WAR in the spreadsheet. And as proof that this doesn’t answer everything, or apply equally to everyone across the board, those Dodgers won 85 games. Based on WAR, they should’ve won 85 games. So they didn’t get a massive hidden bump, but this is why we look at groups of teams, instead of individual teams anecdotally. The short of it is: okay, it seems like there’s a little something extra, at least as far as WAR is concerned. A team with a really strong bullpen might be expected to slightly out-perform its WAR. Meanwhile, a team with a really weak bullpen might be expected to slightly under-perform its WAR. There would be more rigorous ways to look at this, so I don’t want to overstate my own conclusions, but this speaks to possible regular-season effects, before you even consider the benefit of having a strong bullpen once the playoffs roll around. That aspect, we already knew about, when you can just pile the important innings on the best arms you have. The regular-season part feels a little more new, and while the effects don’t appear to be dramatic, remember that a win here or there is important. A win on the free-agent market goes for maybe $8 million, and it’s worth even more if you’re somewhere on the bubble. Of course, you always want to have good relievers. Good relievers make a team better. But they might even make a team more better than other numbers would suggest. Relievers are in. Strong bullpens are in. It’s currently a point of particular emphasis, and while some of that’s just a response to models that’ve been recently successful, that same success has led to some other discoveries. Teams are figuring out the bullpen’s importance. The actual tricky part is the projecting.