Some Early Observations on Reliever Usage
My goal at the start of the research for this article was to develop a quick methodology (i.e. not digging into individual game logs) to find out how effectively teams were deploying their relievers. My first cut was to take a team’s qualifying relievers ranked by gmLI and compare that to the same set of relievers ranked by CHONE ERA. The simple numerical representation I used was the absolute value of the difference between individual pitcher’s rankings summed over each team. For example a player that was ranked 2nd on his team in gmLI, but 3rd in CHONE ERA would have a difference of 1, so a team that has two players flipped would get a total score of 2. So, according to this metric, who is doing well so far? The Rays, Twins and Phillies all have a score of 2, and the next tier is the Cubs Marlins and Orioles with 4. The bottom list consists of the Indians and White Sox at 16, and the Rangers at 14*. Full list can be found here.
That brings me to observation number two, a couple of guys that have already seen their role (read leverage) shift. Randy Williams of the White Sox is ranked 2nd in gmLI and 6th (out of 7) in CHONE ERA for his bullpen, and it seems like Ozzie has figured this out. Here’s a chart of his LI (in this case pLI as it was easier to scrape from the game logs) by appearance
That, my friends, is being put in the reliever doghouse.
On the flip side is Jason Bulger of the Angels. He ranks 4th in gmLI, but 1st in CHONE ERA. Scioscia also seems to be figuring this out as here’s his chart.
I’ll willingly admit that my quick little metric from above might not be the best way to judge a managers bullpen usage, but what do you guys think? Fans of the teams that were in the top or bottom lists, does your team appearing there make sense? Is it worth looking at this retrospectively (i.e. for 2009)?
*I didn’t adjust for Starter to Reliever conversions, so that likely affects the Rangers ranking.
Steve's ramblings about baseball can also be found at Beyond the Box Score and Play a Hard Nine or you can follow him on Twitter
Very cool. I’ll be checking back to see what the data looks like as the year goes on.
I wonder how the use of relievers impacts a team’s performance relative to Pythagenport W/L. Is there a relationship between the “proper” use of relievers and the team’s ability to outperform its Pyth? Also, this would be a cool longterm project, as a way to evaluate managers.
Good stuff Steve. I was actually looking for a similar way to do this a few months back. I tried to use ERA estimator/Team ERA estimator as the ranking system.
I think I’m going to have to do some intra team clustering or something… For some teams there’s not enough differentiation in projected ERAs for the ranking to be significant
This is a great idea and of course will be more accurate and informative when the sample size is larger.
As an Orioles fan it’s hard for me to believe that Dave Trembley has been using his bullpen that effectively. Two explanations for this disjoint come to mind: (1) does restricting your sample to “qualifying” relievers exclude Mike Gonzalez, who has been terrible in extremely high leverage situations but has only appeared in three games? (2) does using CHONE ERA rather than actual ERA make a difference for someone like Jim Johnson, who has an ERA over 7 in high-leverage situations but whose peripheral stats (K, BB, HR) suggest that he’s pitching a lot better than his ERA indicates?
Steve, I am totally stealing these for inter-team comparisons. I am worried they may be cluttered. Let me see what I can do.
Steve, I’ve looked into bullpen usage, too, and I think you’ll also want to include platoon rates. Might be a bit trickier to quantify.
Definitely agree that they would need to be included.