Building a Great Bullpen

There are a number of reasons that the Seattle Mariners find themselves in a heated race for a playoff spot as the 2014 season winds down. Felix Hernandez for one, and Robinson Cano and Hisashi Iwakuma, for two and three. And Kyle Seager, for four. They upgraded both their outfield defense and the back of their rotation from awful to fairly solid, both massive upgrades. On top of it all, their bullpen has by far the best ERA in the major leagues at 2.41, effectively shortening games to six or seven innings. In 2013, the Mariner pen, with many of the same pitchers in place, was not very good. This leads one to a number of questions regarding reliever performance. How does bullpen performance correlate with winning? What have similar frontrunning pens of the past few years had in common? Do high-end pens stay on top for very long?

Thus far in 2014, AL relief pitchers have a cumulative 3.61 ERA, through Tuesday night’s games. The Mariners pen’s 2.41 ERA translates to a 67 Relative Bullpen ERA. We’ll use that number to compare the leading bullpens going back to 2000 below. ERA, of course, isn’t the most cutting-edge statistic out there, and it’s dangerous to use it evaluate individual relievers and their inherent small sample sizes. Adding all of the relievers’ innings together gets the innings count up to 450 and above, making its usage much more palatable.

YR AL REL BPERA NL REL BPERA
2014 SEA 67 SD 74
2013 KC 69 ATL 70
2012 TB 81 CIN 70
2011 NYY 82 ATL 84
2010 TB 85 SD 71
2009 OAK 85 LAD 78
2008 TOR 71 PHL 79
2007 BOS 72 SD 75
2006 MIN 68 NYM 78
2005 CLE 69 STL 75
2004 ANA 82 STL 74
2003 ANA 74 LAD 61
2002 ANA 70 ATL 68
2001 SEA 74 CIN 90
2000 BOS 84 LAD 82

No, we haven’t adjusted for context here. It’s just team bullpen ERA, divided by league bullpen ERA. Considering that fact, you might expect the teams with the most pitcher-friendly ballparks to constantly appear at the top of their league. That’s not the case, however – 17 different organizations have led their respective league in relative bullpen ERA since 2000. The one thing that these clubs do have in common, however, is…..winning. All but four of these 30 clubs posted a winning record in that given season, and as a group they average 92 wins per season. So, having a good bullpen correlates with winning. That tells us a little something, I guess, but a truly good bullpen doesn’t just have significant success in that one peak season. Let’s take a look and see how the very best of these bullpens fared in the season before and after their peak season.

YR TM PREV PEAK NEXT
2003 LAD 93 61 75
2014 SEA 124 67 N/A
2006 MIN 80 68 89
2002 ATL 91 68 98
2013 KC 89 69 93
2005 CLE 115 69 111
2013 ATL 73 70 87
2012 CIN 99 70 94
2002 ANA 86 70 74
2010 SD 94 71 85
2008 TOR 80 71 98

These 11 clubs combined for a relative bullpen ERA of 69 in their respective peak seasons – 31% better than the average bullpen ERA, which is better than the overall league average ERA. Nine of these 11 bullpens (all except the 2005 Indians and 2014 Mariners) had a better than league average bullpen ERA in the season prior to their peak year, and nine of the 10 applicable bullpens (all but those 2005 Indians) had a better than league average bullpen ERA in the season following their peak year. However, very few of these pens were materially better than league average in either the year before or after their peak season. Only the 2013 Braves had a relative bullpen ERA below 80 in the season immediately preceding their peak year, and only the 2002 Angels and 2003 Dodgers did so in the season immediately following their peak year. These 11 top single-season pens had an average relative bullpen ERA of 93 in the year immediately preceding their peak season, and 90 in the year immediately following their peak season.

The 2005 Indians’ pen had the worst three-year run of this group. Bob Wickman was their closer all three seasons, and was often hurt. Their peak year included best-case scenario performances from vets Bobby Howry, David Riske and Arthur Rhodes, all of whom were about to decline in effectiveness and workload. Counting upward, the 2014 Mariners would come next. The big change in their pen this season was the addition of high-end closer Fernando Rodney, which allowed their younger hard throwers to each slide down a role in the pecking order. Still, three of the five most often-used Mariner relievers are the same in both 2013 and 2014 – their actual talent certainly lies somewhere between their exceptional 2014 and awful 2013.

The 2012 Reds featured the breakthrough of Aroldis Chapman into the closer role. That said, arguably the biggest difference between their performance that year compared to 2011 and 2013 was the presence of Sean Marshall in a key setup role. The remainder of their supporting cast was solid throughout the three-year period, but only a healthy Marshall gave them a second difference-maker. The 2002 Braves had a freak season from Chris Hammond (ERA under 1.00 in 76 innings), plus Mike Remlinger’s career year and an out-of-nowhere swansong from Darren Holmes. Just one year later, they featured the likes of Trey Hodges and Jung Bong among their most often used relievers.

The 2013 Royals actually had very similar pen makeup during their three-year period. Greg Holland, Kelvin Herrera and Aaron Crow were among their most often used relievers from 2012-14, with nary an above league average ERA among them. What has bitten them this season is the awful performance of their least-utilized relievers – there’s a bunch of 6 to 7-plus ERAs among that group. The 2010-11 Padres pen was fronted by closer Heath Bell and setup men Luke Gregerson and Mike Adams. Adams was the difference-maker in this group – he wasn’t there in 2009, when his innings went to the likes of Luis Perdomo and Greg Burke, and he was dealt to the Rangers late in 2011.

The 2008 Jays pen performance was driven by lefty Scott Downs. He was one of their most frequently used relievers in the three-year period, but his effectiveness and workload both peaked in 2008, which also saw career years from the likes of Jesse Carlson and Brian Tallet.

So, most of these “great” single-season pens – remember, these are the top 11 of the 450 individual team bullpen-seasons – were largely one-season wonders. These pens were largely solid in the previous and succeeding years, but sustained team bullpen excellence is apparently a very rare thing. Looking back at the first chart of league-leading team pens by season, only two organizations have led their league in consecutive seasons – the 2004-05 Cardinals, who won a total of 205 games, and the 2002-04 Angels, who won the World Series in their peak pen performance season.

Next, let’s take a look at the personnel makeup of the four best of these 11 pens – the ones who performed the best over the three-year span, the midpoint of which is represented by their peak performance season.

TM ROLE NAME ERA – TM NAME ERA – TM NAME ERA –
2005 MIN CL Nathan 62 2006 MIN Nathan 35 2007 MIN Nathan 42
R1 Crain 63 Crain 79 Guerrier 53
R2 Rincon 57 Rincon 65 Neshek 66
R3 Guerrier 78 Guerrier 75 Rincon 115
R4 Mulholland 111 Eyre 119 D.Reyes 90
———- ———– ———— ———– ———- ———— ———– ———- ———— ———–
2013 ATL CL Kimbrel 26 2006 MIN Kimbrel 33 2007 MIN Kimbrel 44
R1 C.Martinez 100 Varvaro 76 Hale 78
R2 Durbin 79 Carpenter 48 Carpenter 93
R3 Venters 82 Avilan 41 Varvaro 76
R4 O’Flaherty 44 Walden 93 Walden 65
———- ———– ———— ———– ———- ———— ———– ———- ———— ———–
2002 ANA CL Percival 60 2006 MIN Percival 43 2007 MIN Percival 78
R1 Pote 93 Weber 57 Rodriguez 68
R2 Levine 53 Levine 96 Weber 60
R3 Weber 77 Pote 73 Donnelly 35
R4 Hasegawa 91 Donnelly 49 Shields 64
———- ———– ———— ———– ———- ———— ———– ———- ———— ———–
2003 LAD CL Gagne 51 2006 MIN Gagne 30 2007 MIN Gagne 53
R1 Carrara 85 Mota 49 Sanchez 81
R2 Quantrill 70 Quantrill 43 Mota 74
R3 Mota 108 Shuey 74 Carrara 52
R4 Mulholland 139 Martin 87 Dreifort 107

These pens are listed from bottom to top, from the #4 2006 Twins (average relative three-year bullpen ERA of 79.0) to the #1 2003 Dodgers (76.3). For each club, their closer and the four pitchers with the next highest relief innings totals are listed, along with their ERA – (ERA relative to league average, not league bullpen average) for each season.

#4 – 2006 Twins (three-year relative bullpen ERA = 79.0)
This was a very stable cast of characters, with an elite closer, Joe Nathan, in the prime of his career, and two key setup men, Juan Rincon and Matt Guerrier, in place for all three seasons. Not a first round pick among this group – the highest draft pick among them was Jesse Crain, a second rounder. This was also a very cost-effective group – they cost under $4.0M cumulatively in 2005, as Crain/Rincon/Guerrier were all 0-3 service time guys. Oh, and Terry Mulholland is here, pitching terribly in 2005. He’ll be back later.

#3 – 2013 Braves (76.7)
Hammer closer in the prime of his career? Check, in the person of Craig Kimbrel. Interestingly, likely the two most talented pitchers of the remainder of this group, Jonny Venters and Eric O’Flaherty, saw their workloads diminish or disappear after the first year of the Braves’ three-year peak period. It’s been a lot of fellow 12th round picks Anthony Varvaro, David Carpenter and Jordan Walden since then, in support of Kimbrel. Their 2013 peak season pen core cost them an incredibly cheap $2.7M, total.

#2 – 2002 Angels (76.7)
Yet another elite closer in his prime, in Troy Percival. Interestingly, it’s only in the year following their peak season that we see their signature bullpen, with Percival, Francisco Rodriguez and Scot Shields listed among their most frequently-used relievers – and the Angels won only 77 games that year. In 2001-02, it was a healthy dose of Ben Weber, Al Levine and Lew Pote. They got a huge boost from late-round draftees – Levine (11th), Weber (20th), Brendan Donnelly (27th), Pote (29th) and Shields (38th) were all selected after the 10th round.

#1 – 2003 Dodgers (76.3)
Eric Gagne was about as dominant as a closer can be over this three-year stretch, both in terms of quality and workload. Their 2003 peak season featured 82 1/3 innings of Gagne and an amazing 105 relief innings from Guillermo Mota, both with ERA- figures below 50. Then there’s the criminally underrated Paul Quantrill, who was dominant over a healthy workload in 2002-03. Throw in a couple fluky seasons from Giovanni Carrara, and you have a historically great three-year run. Oh, and there’s Terry Mulholland again, getting lit as part of an otherwise exceptional pen. There are three first rounders on their list – Mulholland, Paul Shuey and Darren Dreifort – among the most highly compensated but least essential parts of these pens.

So what have we got? A really good pen generally correlates with winning, but a really good pen is more often than not just a bit better than league average in the years before and after its peak. There is heavy year-to-year turnover in most pens, and in even the best of them, the draft pedigree of the individual members often isn’t that great. The very best pens are quite often very inexpensive, featuring multiple cost-controlled 0-3 years players. A hammer, top-of-his-game closer seems to be a necessity, but more often than not, an elite bullpen features an elite closer who isn’t yet paid elite dollars.

Let’s take this back full circle to this year’s elite pen, which belongs to the Seattle Mariners. Rodney might not be Kimbrel, but he’s a top-shelf closer. The rest of the guys have pedigrees resembling those of many of the pitchers we’ve discussed here. Danny Farquhar is a well-traveled 40-man roster bubble guy. Dominic Leone was a 16th round pick. Tom Wilhelmsen was out of baseball for several years, showed up in Arizona for a tryout, and worked his way to the big leagues.

The going rate for free agent closers has come down to reasonable levels in recent offseasons, and teams that long avoided that market are now – wisely – back in. Putting significant dollars into anything less than a high-end closer? The tables above should give clubs pause from going there. Short-term, high-end relievers come from everywhere – from the late rounds of the draft, relatively low-dollar Latin American signings, waiver claims, independent leagues, minor trades, etc.. These pitchers are usually starters as amateurs, but move to the pen as professionals. They generally have above average fastballs, a plus secondary pitch and/or a distinguishing batted-ball profile characteristic, i.e., a very high grounder or popup rate.

During my tenure with the Mariners, I was significantly involved in the amateur draft. In 2011, we took Carson Smith in the 8th round, and in 2012, took Leone in the 16th. Smith was a prime target of mine – size, downplane, velocity, workload, and numbers – a huge grounder rate. Leone was a prime target of our Scouting Director, Tom McNamara, who in addition to admiring Leone’s measurables, had a gut feel for the kid’s heart. Our area scouts and crosscheckers loved these kids, and had all of their ducks in a row, paving the way for their eventual selection.

Leone’s had a strong rookie season, and Smith has been lights out in his first MLB outings, in the heat of a pennant race. As we have seen above, you need a steady stream of these guys to have a strong bullpen on an ongoing basis, and you don’t have to snag them in the early rounds of the draft, or pay full price for anything but a high-end closer on the free agent market. Individual relievers will break your heart, but a sound organizational strategy geared toward the accumulation of a critical mass of potential major league relievers is pivotal to building a winning organization.

We hoped you liked reading Building a Great Bullpen by Tony Blengino!

Please support FanGraphs by becoming a member. We publish thousands of articles a year, host multiple podcasts, and have an ever growing database of baseball stats.

FanGraphs does not have a paywall. With your membership, we can continue to offer the content you've come to rely on and add to our unique baseball coverage.

Support FanGraphs




newest oldest most voted
fred
Guest
fred

Do the great bullpens have a certain L/R profile? I noticed a lot of right-handers.

Zac
Guest
Zac

I think that most bullpens will be constructed in a fairly balanced mix of L/R handed guys. A lot of the best ‘pens will have one guy that can go long (think ’14 Wilhemsen) a couple of lefty/righty specialists that have lights out splits (Furbush) and then a set-up (Medina/Farqhuar), and a closer (Rodney)