Building a Great Bullpen

There are a number of reasons that the Seattle Mariners find themselves in a heated race for a playoff spot as the 2014 season winds down. Felix Hernandez for one, and Robinson Cano and Hisashi Iwakuma, for two and three. And Kyle Seager, for four. They upgraded both their outfield defense and the back of their rotation from awful to fairly solid, both massive upgrades. On top of it all, their bullpen has by far the best ERA in the major leagues at 2.41, effectively shortening games to six or seven innings. In 2013, the Mariner pen, with many of the same pitchers in place, was not very good. This leads one to a number of questions regarding reliever performance. How does bullpen performance correlate with winning? What have similar frontrunning pens of the past few years had in common? Do high-end pens stay on top for very long?

Thus far in 2014, AL relief pitchers have a cumulative 3.61 ERA, through Tuesday night’s games. The Mariners pen’s 2.41 ERA translates to a 67 Relative Bullpen ERA. We’ll use that number to compare the leading bullpens going back to 2000 below. ERA, of course, isn’t the most cutting-edge statistic out there, and it’s dangerous to use it evaluate individual relievers and their inherent small sample sizes. Adding all of the relievers’ innings together gets the innings count up to 450 and above, making its usage much more palatable.

YR	AL	REL BPERA	NL	REL BPERA
2014	SEA	67	SD	74
2013	KC	69	ATL	70
2012	TB	81	CIN	70
2011	NYY	82	ATL	84
2010	TB	85	SD	71
2009	OAK	85	LAD	78
2008	TOR	71	PHL	79
2007	BOS	72	SD	75
2006	MIN	68	NYM	78
2005	CLE	69	STL	75
2004	ANA	82	STL	74
2003	ANA	74	LAD	61
2002	ANA	70	ATL	68
2001	SEA	74	CIN	90
2000	BOS	84	LAD	82

No, we haven’t adjusted for context here. It’s just team bullpen ERA, divided by league bullpen ERA. Considering that fact, you might expect the teams with the most pitcher-friendly ballparks to constantly appear at the top of their league. That’s not the case, however – 17 different organizations have led their respective league in relative bullpen ERA since 2000. The one thing that these clubs do have in common, however, is…..winning. All but four of these 30 clubs posted a winning record in that given season, and as a group they average 92 wins per season. So, having a good bullpen correlates with winning. That tells us a little something, I guess, but a truly good bullpen doesn’t just have significant success in that one peak season. Let’s take a look and see how the very best of these bullpens fared in the season before and after their peak season.

YR	TM	PREV	PEAK	NEXT
2003	LAD	93	61	75
2014	SEA	124	67	N/A
2006	MIN	80	68	89
2002	ATL	91	68	98
2013	KC	89	69	93
2005	CLE	115	69	111
2013	ATL	73	70	87
2012	CIN	99	70	94
2002	ANA	86	70	74
2010	SD	94	71	85
2008	TOR	80	71	98

These 11 clubs combined for a relative bullpen ERA of 69 in their respective peak seasons – 31% better than the average bullpen ERA, which is better than the overall league average ERA. Nine of these 11 bullpens (all except the 2005 Indians and 2014 Mariners) had a better than league average bullpen ERA in the season prior to their peak year, and nine of the 10 applicable bullpens (all but those 2005 Indians) had a better than league average bullpen ERA in the season following their peak year. However, very few of these pens were materially better than league average in either the year before or after their peak season. Only the 2013 Braves had a relative bullpen ERA below 80 in the season immediately preceding their peak year, and only the 2002 Angels and 2003 Dodgers did so in the season immediately following their peak year. These 11 top single-season pens had an average relative bullpen ERA of 93 in the year immediately preceding their peak season, and 90 in the year immediately following their peak season.

The 2005 Indians’ pen had the worst three-year run of this group. Bob Wickman was their closer all three seasons, and was often hurt. Their peak year included best-case scenario performances from vets Bobby Howry, David Riske and Arthur Rhodes, all of whom were about to decline in effectiveness and workload. Counting upward, the 2014 Mariners would come next. The big change in their pen this season was the addition of high-end closer Fernando Rodney, which allowed their younger hard throwers to each slide down a role in the pecking order. Still, three of the five most often-used Mariner relievers are the same in both 2013 and 2014 – their actual talent certainly lies somewhere between their exceptional 2014 and awful 2013.

The 2012 Reds featured the breakthrough of Aroldis Chapman into the closer role. That said, arguably the biggest difference between their performance that year compared to 2011 and 2013 was the presence of Sean Marshall in a key setup role. The remainder of their supporting cast was solid throughout the three-year period, but only a healthy Marshall gave them a second difference-maker. The 2002 Braves had a freak season from Chris Hammond (ERA under 1.00 in 76 innings), plus Mike Remlinger’s career year and an out-of-nowhere swansong from Darren Holmes. Just one year later, they featured the likes of Trey Hodges and Jung Bong among their most often used relievers.

The 2013 Royals actually had very similar pen makeup during their three-year period. Greg Holland, Kelvin Herrera and Aaron Crow were among their most often used relievers from 2012-14, with nary an above league average ERA among them. What has bitten them this season is the awful performance of their least-utilized relievers – there’s a bunch of 6 to 7-plus ERAs among that group. The 2010-11 Padres pen was fronted by closer Heath Bell and setup men Luke Gregerson and Mike Adams. Adams was the difference-maker in this group – he wasn’t there in 2009, when his innings went to the likes of Luis Perdomo and Greg Burke, and he was dealt to the Rangers late in 2011.

The 2008 Jays pen performance was driven by lefty Scott Downs. He was one of their most frequently used relievers in the three-year period, but his effectiveness and workload both peaked in 2008, which also saw career years from the likes of Jesse Carlson and Brian Tallet.

So, most of these “great” single-season pens – remember, these are the top 11 of the 450 individual team bullpen-seasons – were largely one-season wonders. These pens were largely solid in the previous and succeeding years, but sustained team bullpen excellence is apparently a very rare thing. Looking back at the first chart of league-leading team pens by season, only two organizations have led their league in consecutive seasons – the 2004-05 Cardinals, who won a total of 205 games, and the 2002-04 Angels, who won the World Series in their peak pen performance season.

Next, let’s take a look at the personnel makeup of the four best of these 11 pens – the ones who performed the best over the three-year span, the midpoint of which is represented by their peak performance season.

TM	ROLE	NAME	ERA –	TM	NAME	ERA –	TM	NAME	ERA –
2005 MIN	CL	Nathan	62	2006 MIN	Nathan	35	2007 MIN	Nathan	42
	R1	Crain	63		Crain	79		Guerrier	53
	R2	Rincon	57		Rincon	65		Neshek	66
	R3	Guerrier	78		Guerrier	75		Rincon	115
	R4	Mulholland	111		Eyre	119		D.Reyes	90
———-	———–	————	———–	———-	————	———–	———-	————	———–
2013 ATL	CL	Kimbrel	26	2006 MIN	Kimbrel	33	2007 MIN	Kimbrel	44
	R1	C.Martinez	100		Varvaro	76		Hale	78
	R2	Durbin	79		Carpenter	48		Carpenter	93
	R3	Venters	82		Avilan	41		Varvaro	76
	R4	O’Flaherty	44		Walden	93		Walden	65
———-	———–	————	———–	———-	————	———–	———-	————	———–
2002 ANA	CL	Percival	60	2006 MIN	Percival	43	2007 MIN	Percival	78
	R1	Pote	93		Weber	57		Rodriguez	68
	R2	Levine	53		Levine	96		Weber	60
	R3	Weber	77		Pote	73		Donnelly	35
	R4	Hasegawa	91		Donnelly	49		Shields	64
———-	———–	————	———–	———-	————	———–	———-	————	———–
2003 LAD	CL	Gagne	51	2006 MIN	Gagne	30	2007 MIN	Gagne	53
	R1	Carrara	85		Mota	49		Sanchez	81
	R2	Quantrill	70		Quantrill	43		Mota	74
	R3	Mota	108		Shuey	74		Carrara	52
	R4	Mulholland	139		Martin	87		Dreifort	107

These pens are listed from bottom to top, from the #4 2006 Twins (average relative three-year bullpen ERA of 79.0) to the #1 2003 Dodgers (76.3). For each club, their closer and the four pitchers with the next highest relief innings totals are listed, along with their ERA – (ERA relative to league average, not league bullpen average) for each season.

#4 – 2006 Twins (three-year relative bullpen ERA = 79.0)
This was a very stable cast of characters, with an elite closer, Joe Nathan, in the prime of his career, and two key setup men, Juan Rincon and Matt Guerrier, in place for all three seasons. Not a first round pick among this group – the highest draft pick among them was Jesse Crain, a second rounder. This was also a very cost-effective group – they cost under $4.0M cumulatively in 2005, as Crain/Rincon/Guerrier were all 0-3 service time guys. Oh, and Terry Mulholland is here, pitching terribly in 2005. He’ll be back later.

#3 – 2013 Braves (76.7)
Hammer closer in the prime of his career? Check, in the person of Craig Kimbrel. Interestingly, likely the two most talented pitchers of the remainder of this group, Jonny Venters and Eric O’Flaherty, saw their workloads diminish or disappear after the first year of the Braves’ three-year peak period. It’s been a lot of fellow 12th round picks Anthony Varvaro, David Carpenter and Jordan Walden since then, in support of Kimbrel. Their 2013 peak season pen core cost them an incredibly cheap $2.7M, total.

#2 – 2002 Angels (76.7)
Yet another elite closer in his prime, in Troy Percival. Interestingly, it’s only in the year following their peak season that we see their signature bullpen, with Percival, Francisco Rodriguez and Scot Shields listed among their most frequently-used relievers – and the Angels won only 77 games that year. In 2001-02, it was a healthy dose of Ben Weber, Al Levine and Lew Pote. They got a huge boost from late-round draftees – Levine (11th), Weber (20th), Brendan Donnelly (27th), Pote (29th) and Shields (38th) were all selected after the 10th round.

#1 – 2003 Dodgers (76.3)
Eric Gagne was about as dominant as a closer can be over this three-year stretch, both in terms of quality and workload. Their 2003 peak season featured 82 1/3 innings of Gagne and an amazing 105 relief innings from Guillermo Mota, both with ERA- figures below 50. Then there’s the criminally underrated Paul Quantrill, who was dominant over a healthy workload in 2002-03. Throw in a couple fluky seasons from Giovanni Carrara, and you have a historically great three-year run. Oh, and there’s Terry Mulholland again, getting lit as part of an otherwise exceptional pen. There are three first rounders on their list – Mulholland, Paul Shuey and Darren Dreifort – among the most highly compensated but least essential parts of these pens.

So what have we got? A really good pen generally correlates with winning, but a really good pen is more often than not just a bit better than league average in the years before and after its peak. There is heavy year-to-year turnover in most pens, and in even the best of them, the draft pedigree of the individual members often isn’t that great. The very best pens are quite often very inexpensive, featuring multiple cost-controlled 0-3 years players. A hammer, top-of-his-game closer seems to be a necessity, but more often than not, an elite bullpen features an elite closer who isn’t yet paid elite dollars.

Let’s take this back full circle to this year’s elite pen, which belongs to the Seattle Mariners. Rodney might not be Kimbrel, but he’s a top-shelf closer. The rest of the guys have pedigrees resembling those of many of the pitchers we’ve discussed here. Danny Farquhar is a well-traveled 40-man roster bubble guy. Dominic Leone was a 16th round pick. Tom Wilhelmsen was out of baseball for several years, showed up in Arizona for a tryout, and worked his way to the big leagues.

The going rate for free agent closers has come down to reasonable levels in recent offseasons, and teams that long avoided that market are now – wisely – back in. Putting significant dollars into anything less than a high-end closer? The tables above should give clubs pause from going there. Short-term, high-end relievers come from everywhere – from the late rounds of the draft, relatively low-dollar Latin American signings, waiver claims, independent leagues, minor trades, etc.. These pitchers are usually starters as amateurs, but move to the pen as professionals. They generally have above average fastballs, a plus secondary pitch and/or a distinguishing batted-ball profile characteristic, i.e., a very high grounder or popup rate.

During my tenure with the Mariners, I was significantly involved in the amateur draft. In 2011, we took Carson Smith in the 8th round, and in 2012, took Leone in the 16th. Smith was a prime target of mine – size, downplane, velocity, workload, and numbers – a huge grounder rate. Leone was a prime target of our Scouting Director, Tom McNamara, who in addition to admiring Leone’s measurables, had a gut feel for the kid’s heart. Our area scouts and crosscheckers loved these kids, and had all of their ducks in a row, paving the way for their eventual selection.

Leone’s had a strong rookie season, and Smith has been lights out in his first MLB outings, in the heat of a pennant race. As we have seen above, you need a steady stream of these guys to have a strong bullpen on an ongoing basis, and you don’t have to snag them in the early rounds of the draft, or pay full price for anything but a high-end closer on the free agent market. Individual relievers will break your heart, but a sound organizational strategy geared toward the accumulation of a critical mass of potential major league relievers is pivotal to building a winning organization.