Fun with Shutdowns and Meltdowns
Yesterday David introduced a couple of new stats, Shutdowns and Meltdowns, to the site. It’s fashioned after saves/blown saves but is vastly superior, because it’s a metric that uses WPA as a substitute of the brainless, archaic save stat and the rules that guide it.
A team essentially has something like a 98% likelihood of winning the game with a three-run lead with no outs, yet a manager will trot out his ace reliever in that situation about 98% of the time for the sake of save. But when the game is on the line and it’s non-save situation, we often see managers make some of the most bizarre choices in their bullpen usage.
Take for instance Tuesday night’s Phillies–Cardinals game that ended in the 10th on a walk-off homer by Carlos Ruiz. While there’s no real “ace” in the Cardinal bullpen, the inexperienced Blake Hawksworth isn’t the guy you normally would want on the mound against the Phillies in such a high leverage situation, but it appears Tony La Russa held back his closer because it wasn’t a save situation.
Anyway, with any luck this catches on. Just to recap:
A Shutdown is when a reliever accumulates greater than or equal to 0.06 WPA in any individual game.
A Meltdown is when a reliever’s WPA is less than or equal to -0.06 in any individual game.
What I thought would be interesting is to look at the “Meltdowniest” pitchers of the past three seasons, as well as the ones who we could say have ice water in their veins. The pitchers with the most meltdowns are usually the ones fans want to ride out of town on a rail, along with their manager, while the pitchers with the smallest meltdown rate we tend to feel pretty comfortable with, even in the highest leverage situations.
These are the pitchers with the highest percentage of relief appearances that resulted in a Meltdown:
| Name | G | SD | MD | Meltdown% |
| Aaron Heilman | 229 | 55 | 52 | 22.7% |
| Luis Ayala | 163 | 33 | 37 | 22.7% |
| Shawn Camp | 149 | 36 | 33 | 22.1% |
| Brian Bass | 93 | 18 | 20 | 21.5% |
| Juan Rincon | 143 | 27 | 30 | 21.0% |
| Scott Linebrink | 178 | 61 | 37 | 20.8% |
| Sean Green | 215 | 52 | 43 | 20.0% |
| Mark Lowe | 136 | 47 | 27 | 19.9% |
| Zach Miner | 111 | 39 | 22 | 19.8% |
| Cla Meredith | 217 | 45 | 43 | 19.8% |
Aaron Heilman is our King of Catastrophe. Cub fans are glad to see him take his act elsewhere. That four-year, $19 million deal given to Scott Linebrink was given by the White Sox is looking like one the more awful contracts given to a middle reliever in recent memory.
So what pitchers have ice water in their veins, or in other words, the pitchers who have experienced the fewest rate of blowups per appearance? I excluded anyone who didn’t average at least an average pLI of 1.2.
| Name | G | SD | MD | SD% | Meltdown% |
| Joe Nathan | 206 | 104 | 18 | 50.5% | 8.7% |
| Jonathan Papelbon | 192 | 85 | 17 | 44.3% | 8.9% |
| Joakim Soria | 172 | 86 | 16 | 50.0% | 9.3% |
| Mariano Rivera | 197 | 104 | 20 | 52.8% | 10.2% |
| Takashi Saito | 164 | 57 | 17 | 34.8% | 10.4% |
| Jose Valverde | 191 | 89 | 21 | 46.6% | 11.0% |
| Francisco Cordero | 206 | 101 | 23 | 49.0% | 11.2% |
| David Aardsma | 145 | 63 | 17 | 43.4% | 11.7% |
| Rafael Soriano | 162 | 63 | 19 | 38.9% | 11.7% |
| Bobby Jenks | 175 | 76 | 21 | 43.4% | 12.0% |
No major surprises here, especially the from the top four.
What about the highest Meltdown % by pitchers with 30 or more saves the past three seasons?
| Name | G | SD | MD | SD% | Meltdown% |
| Chad Qualls | 207 | 88 | 36 | 42.5% | 17.4% |
| David Weathers | 210 | 73 | 36 | 34.8% | 17.1% |
| Kevin Gregg | 218 | 89 | 35 | 40.8% | 16.1% |
| Brian Wilson | 155 | 84 | 24 | 54.2% | 15.5% |
| C.J. Wilson | 190 | 74 | 29 | 38.9% | 15.3% |
| Fernando Rodney | 159 | 66 | 24 | 41.5% | 15.1% |
| Matt Capps | 182 | 75 | 26 | 41.2% | 14.3% |
| Brad Lidge | 205 | 83 | 29 | 40.5% | 14.1% |
| Brandon Lyon | 199 | 78 | 28 | 39.2% | 14.1% |
| Trevor Hoffman | 164 | 86 | 23 | 52.4% | 14.0% |
The unpopular trio of Chad Qualls, Kevin Gregg and David “Stormy” Weathers lead the list. We also see the Phillies favorite bipolar pitcher, Brad Lidge make this list. Interestingly enough, Brian Wilson had the highest percentage of his outings end in either a Shutdown or a Meltdown of any reliever.
Again, I hope this catches on. Shutdowns and Meltdowns are a more practical and intuitive way at evaluating reliever performance, and I think a more fun one, too.
Erik Manning is the founder of Future Redbirds and covers the Cardinals for Heater Magazine. You can get more of his analysis and rantings in bite-sized bits by following him on twitter.
This is actually kind of interesting. I sort of expected Kevin Gregg to show up in your third list, but I’m actually kind of surprised to see Trevor Hoffman in there too. What about a SD to MD ratio? Maybe it’s not just important to see how many times in general someone “shuts down”, but it’s also important to see what the ratio of great games to bad games is.
Like I suggested elsewhere, though, I’d like to scale WPA to outs to be able to better estimate the impact of the usage of lesser relievers in smaller situations. How much does a LOOGY contribute to the bullpen when used specifically in those situations? Perhaps a secondary statistic based on SD can be devised that measures the effectiveness of relievers when used in lower-out situations, or SD can be expanded.
So really, why not scale the amount of WPA added to the amount of outs the pitcher gets? If you get a positive WPA in just one out, that means you’re doing your job as a LOOGY, right? You can set the tolerance for this stat pretty low because it wants to measure minute changes.
We know that the +/- .06 WPA is essentially over three outs. What about +/- .04 WPA for a 2 out situation and +/- .02 WPA for a 1 out situation? This is obviously just throwing an idea out there, because I’m not (yet) a statistician.
A LOOGY should be able to still get a shutdown if he comes in for one or two outs in certain situations, especially if it’s a close game and there are runners on base. The WPA changes enough in close games in the later innings that if the LOOGY comes in, gets one or two outs, he should still be able to get a shutdown even though he didn’t pitch three outs.
But if a LOOGY comes in to get a single out earlier in the game when the score isn’t close, Im not sure that reliever has earned a shutdown, as I’m not sure how much that really helped the team get closer to winning
Yeah, take for example the Cards. Their LOOGYs are 2nd and 3rd in SDs because they have a decent amount of appearances in high lev situations
However, I do think the SD/MD ratio could be very useful.
It might be interesting because it would be good to quickly rate the consistency — and value — of people with decent shutdown percentages at a glance.
Case in point: In the past three seasons, Kevin Gregg has an SD/MD of 2.54, Joe Nathan has an SD/MD of 5.78 and Trevor Hoffman has an SD/MD of 3.74. You already knew Nathan was better than Kevin Gregg anyway, but it’s nice to have a quick metric that tells you what’s up. You can also obtain an average league SD/MD and quickly determine who is above or below average in this aspect.
Honestly, I originally thought of looking at it this way, but for some reason or another I settled the percentages of their games that were either shutdowns/meltdowns or neutral. Looking back, I think I’d just do the SD/MD for all the reasons stated. Maybe for another post I can span SD/MD ratios over the decades, see who was the king of each categories for aughts, 90’s, 80’s…
Of course, I listed the SD/MDs in the tables so you can still eyeball it.
Erik, it’s not like you can’t have both. It’s not a bad idea to be able to see how many games a reliever came into in which they didn’t really wildly affect the outcome of the game, so maybe having SD, MD, NE (neutral effect) and SD/MD could be the four things we’re looking to determine.
I agree having both would be good to see, as the SD% and MD% show how often the pitcher is used during important situations in the game. The SD/MD could be a better indication of performance overall.
The win value of an out, when LI = 1, is +.027 wins. So, if you can get two outs when the LI is 1.2, you get your shutdown. To get it for one out, you need the LI to be 2.3.
So, definitely possible to be a LOOGY and earn a shutdown under the current construction.
So in other words, Renyel Pinto’s SD/MD must be like .1 or something