Shutdowns & Meltdowns
This week there’s been a lot of discussion on The Book Blog about creating a save style metric based on WPA. The end result of the discussion was to create two stats: Shutdowns and Meltdowns.
A Shutdown is when a reliever accumulates greater than or equal to 0.06 WPA in any individual game.
A Meltdown is when a reliever’s WPA is less than or equal to -0.06 in any individual game.
The entire discussion of Shutdowns and Meltdowns was started off by Jeff Zimmerman’s “saves rant” over at Beyond the Box Score and then the creation of the new metric was hashed out on The Book Blog here and here.
The number of Shutdowns are scaled to Saves + Holds, which is where the .06 thresholds originated.
Both these metrics: Shutdowns (SD) and Meltdowns (MD) are available in the Win Probability leaderboards and teams sections and will eventually make it into the player pages. Remember you will either need to click on “All Players” or “Relievers” since starting pitchers are not eligible for Shutdowns or Meltdowns.
David Appelman is the creator of FanGraphs.
I just orgasm-ed.
There will probably be some tweaking and study on this in the future, but I wouldnt be surprised if this stat becomes very popular in the future.
There was also some discussion over at Royals Review on this subject (and you get to see how amazing the Royals bullpen really is)
http://www.royalsreview.com/2010/5/5/1443967/shutdowns-and-meltdowns
Fantastic. I’m excited to see how this plays out.
Just read the comments about the scale. Seems interesting.
This is cool. Looking at the team comparisons, the Royals have — shocker — a bad bullpen. Also, most teams’ bullpens have more SD than MD, even if they have negative WPA.
Why did this get pushed out so fast?
It was easy to do and I really liked the idea.
What better way to test it than for Fangraphs to do all the hard work, and then let analysts and fans sink their teeth in it?
If it turns out to be cr-p, it can always be removed.
Would you rather that Fangraphs create a beta site that has extra features until the Fangraphs board approves it for production use?
Couldn’t agree more.
How much time did you spend choosing your user name?
I think you just got a Shutdown with that +6
I feel like the “Meltdown” needs to be a bit more stringent, though. It’s really easy for a guy pitching in a high leverage situation to be worth less than -.06, whereas a guy pitching with a lead might have a tougher time of reaching +.06.
For example, Brian Fuentes yesterday pitched darn well, allowed a solo HR in a game his team was already trailing, and earned a -.056. A meltdown is more like what Chris Perez yesterday, where he was at -.596. Yesterday, Mark Hendrickson came in and pitched rather well, but since there were two men on base and the first hitter he faced doubled, he ends up at -.065, so he suffered a “Meltdown.” The description doesn’t seem fitting since the started didn’t even make it out of the fourth and his team wasn’t leading.
There’s 1.6 times as many shutdowns as meltdowns.
Now, if you are arguing that the terms are not really what it is, then fine, we can debate whether to call things “quality starts” or “not-bad starts” or a “Johnson game”. Feel free to choose better names if you like.
If you don’t like the requirements to earn each of these things, then fine, bring something to the table, rather than saying why things don’t work, like earning a “Win” even though you give up 7 runs is not really earning a win yourself.
Give me something to work with.
Did you run the numbers to scale Meltdowns to blown saves? I expect that the threshold would be a little higher.
No, the meltdowns is the opposite of a shutdown, so the .06 threshhold is the same. I can see how we can change the meltdown threshhold.
As it is, it’s 1.6 to 1. We can change that. It kinda works now, as you get a 1:1 ratio for bad relievers. So, it’s nice that the “Replacement level” is that. You can just do a straight shut minus melt if you like.
so the 1.6:1 shutdowns to meltdowns ratio means that a “replacement” level reliever will have a 1:1 ratio? And rafael perez has a 1:5 ratio. Neat.
Jon, right, that’s what it looks like when you look at the results of the past 3 years.
In general (if we included starters, that is) shutdowns and meltdowns would be a 1:1 ratio, yes (since the “average” pitcher would have a WPA of zero)? So is the change to 1.6:1 a product of relievers generally pitching better than starters (for all the obvious reasons)?
I just want to make sure I understand why 1:1 is replacement rather than “average”.
Thanks.
somehow I don’t think tango took that look to pick his username.
and also, I thought this was going to be about Milton Bradley. shux.
I think he was talking about ogZayYsj3r7CGsz.
How someone could remember that name, I have no idea.
I chose it because it’s my bank password.
What?
Oh, curses…you were talking to ogZayYsj3r7CGsz. I always make that mistake.
There are more than a couple who will be relieved to hear that.
(Relieved? heh.)
Can Ryan Madson get a ‘bonus’ meltdown for taking himself off the roster for a third of the season?
Wouldn’t that be a shutdown?
Operation Shutdown?
Just read about this on Crashburn Alley and then found it here too. Here’s my one question:
Instead of setting a mark at +/- .06 WPA, why not just make a shutdown any time appearance where the pitcher increases his team’s chances of winning at all (and a meltdown the opposite)? It seems that the +/- .06 WPA mark would make it so low-leverage relievers (and probably LOOGYs) could really only accumulate meltdowns. If that assumption is true, you’d have the same problem as with saves/blown saves, just with a different set of relievers. I expect this would lead to an even greater disparity between meltdowns and shutdowns, but I don’t see why that should really have to matter.
Check out the results first, and see how it looks to you. You may be right, but let’s work with data now that Fangraphs has it.
I’ll preface this by saying I don’t know a ton about WPA, so I’m sorry if there are any egregious mistakes in here. Anyway, here’s an example which I think illustrates the point I’m trying to make.
As I understand it, one of the biggest flaws in the saves statistic is that non-closers are unable to pick up saves (unless they pitch the ninth) but are able to pick up blown saves. However, because of the .06 threshold, it is sometimes only possible for a reliever to pick up either a shutdown or a meltdown. For instance, on April 16th, David Herndon entered game between the Phillies and the Marlins where the Phillies had a 99.7% chance of winning the game. Therefore he had no chance of picking up a shutdown, but managed to pick up a meltdown (thanks to some awful luck), as he left with the Phillies chances of winning below 91%.
Now compare two similar situations. Yesterday, the Rangers trailed the Royals by one in the bottom of the 8th with two outs and nobody on. Three days ago, the Astros trailed the D’backs by one in the bottom of the 8th with two outs and nobody on. In the first situation, Soria gave up a home run to the next hitter, giving Soria a -.319 WPA for that play alone. In the other situation, Juan Gutierrez induced a fly out from Pedro Feliz, giving Gutierrez a +.031 WPA for that play. The starting situations are very similar, but you can see that, against one hitter, the worst case scenario pushes the pitcher over the meltdown threshold, while the best case scenario fails to push the pitcher over the shutdown threshold. In other words, one batter can be easily enough to get a meltdown, but except in very high leverage situations, it’s necessary to face more than one batter to get a shutdown. Neither Soria nor Gutierrez is a LOOGY, but LOOGYs are often only sent in to face one or two hitters (e.g. In 40% of his appearances last year, Scott Eyre faced 2 or fewer hitters). This means that, for LOOGYs, a large percentage of their appearances give them a chance to pick up only a meltdown.
You can make the point that low leverage pitchers are used in low leverage situations for a reason, or that LOOGYs are only LOOGYs for a reason, but this stat seems biased against these kinds of pitchers. A LOOGY can face 20 batters in 20 appearances, retired 19 of 20, and end up with 1 meltdown and 0 shutdowns. A mop-up guy can do his job every time and then get bad luck once and end up with 1 meltdown and 0 shutdowns. It seems to me that this stat has the same problem as saves, just on a different level.
The goal is to see how often a reliever does his job in high leverage situations, right? Why not set a leverage threshold instead (no idea what it would be) and then award a player a shutdown for recording a positive WPA in a high leverage situation and a meltdown for recording a negative WPA in a high leverage situation?
Well, here’s an idea. Why not scale the amount of WPA added to the amount of outs the pitcher gets? If you get a positive WPA in just one out, that means you’re doing your job as a LOOGY, right? You can set the tolerance for this stat pretty low because it wants to measure minute changes.
We know that the +/- .06 WPA is essentially over three outs. What about +/- .04 WPA for a 2 out situation and +/- .02 WPA for a 1 out situation? This is obviously just throwing an idea out there, because I’m not (yet) a statistician.
I like this idea a lot more, because it prevents number of batters faced from becoming an issue. The low leverage situation still seems like a problem to me, but I feel like I’m missing something in that case.
“I expect this would lead to an even greater disparity between meltdowns and shutdowns, but I don’t see why that should really have to matter.”
In case it was unclear, by “this,” I meant my positive/negative WPA suggestion.
Instead of a “meltdown” can we just call it a “Linebrink?”
+1 Fastball with no movement
The Farnsworth?
I spent all day packing and moving and then come back to find a nice stat based on how pissed off I was on how the Royals were using Soria.
I used Shutdowns and Meltdowns to come up with a stat called ROL (short for Rolaids because Rolaids spells relief). The formula is (IP*SD*gmLI)/(GM*MD*xFIP), where IP is innings, SD is Shutdowns, gmLI is leverage index entering a game, GM is games, MD is Meltdowns, and xFIP is expected field independent pitching. I ran it for 2009. You can check it out if you want… http://www.themartinezreport.com/2010/05/rolaids-spells-relief-2009.html I have no idea if it has any merit.
I found it entertaining, Andrew. I specifically wanted to see how Nick Masset’s strong 2009 was valued, and it seems in line with his ancedotal value as he provides a ton of good relief innings. Badenhop wasn’t really a surprise to me, as I’m impressed every time I see him pitch. Tons of movement on his pitches and he’s a strike-thrower.
I’ll be interested in the 2010 ROLs.