All About Clutch
Amongst the several great win probability statistics kept here is one simply titled ‘clutch.’ The number measures how well players perform in previously defined clutch situations relative to how they would have performed in a context-neutral environment. It has confused some and come into question from others recently so I thought I would take this time to break it down and try to clear up any confusion or doubts.
The stat is calculated by subtracting the WPA/LI from the WPA/pLI. Now, WPA/LI is an already calculated measure freely available all throughout this site. WPA/pLI, however, would have to be manually calculated by dividing the overall WPA by the average leverage index. As an example let’s use Pat Burrell and his current numbers. Burrell has the third best clutch score in the game at 1.35. He has a WPA/LI of 2.51, a WPA of 4.08, and a pLI of 1.06.
4.08/1.06 = 3.85 and 3.85-2.51 = 1.34. The 1.34 vs. 1.35 is nothing more than a rounding discrepancy. This measures how much better Burrell performed in high leverage situations than all others. If he posted a .900 OPS in crucial plate appearances but an equal OPS in all others, he is not considered clutch. And why should he be? Sure, he posted great numbers in high LI game states but he did not raise his game at all.
This brings me to the first major point: Clutch has different definitions and to understand this statistic we need to be on the same page. No matter how important the media makes clutch performance out to be, it does not refer to performing well with the game on the line. Instead, it refers to performing well in these types of situations relative to all others. The statistic can be summed up by the question, “Does the player raise his game in important situations?” If not, he is not clutch, no matter how great his numbers are in high leverage plate appearances.
The second major point is that being clutch or not being clutch is NOT the same as being good or not being good. You do not need to raise your game in crucial situations to be a great player and those who do raise their games are not necessarily the most talented. A player with a .200 BA that hits .300 in crucial situations is, and should be, considered more clutch than someone with a .333 BA in all situations. The .333 is a better BA but it is not clutch because it did not constitute a raising of the game.
As I pointed out this morning, just 3 of the 33 NL MVP winners from 1974-2007 finished in the top ten in clutch. Barry Bonds, who won the award from 2001-2004, had clutch scores ranging from -0.49 to -1.14 from 2001-2003, and I better not hear anybody discuss those seasons not being insanely productive. His negative clutch score just means that he did not post a 1.980 OPS (exaggeration) in high leverage situations. His high leverage OPS was likely higher than everyone else’s but this statistic works to measure a player against himself since, after all, clutch refers to raising your individual game, no matter how high that game generally turns out.
I hope this clears up some confusion but I have a feeling the vast differences in definitions of this skill/phenomenon/whatever you call it will continue to generate confusion. The media has relied on clutch to the point that we are now mistaking it for good or bad performance. This is incorrect. Clutch means raising your game, not being a good player.
Eric is an accountant and statistical analyst from Philadelphia. He also covers the Phillies at Phillies Nation and can be found here on Twitter.
IN reference to the paragraph that begins, “The second major point is that being clutch or not being clutch is NOT the same as being good or not being good…. ”
This seems to makes the statistic relatively pointless. (The following are all entirely made up just to make a point) Anyone would take a “clutch”-less Albert Pujols because he hits .400 in high leverage situations, but his overall BA was .350 compared to, I don’t know, Hideki Matsui who bats .310 overall, but .370 in clutch situations. It is interesting to see “who can raise their game”, but it does not really tell us anything if it is not put into some greater, league relative context.
I think we look to statistics as a tool to better understand how players have performed and will perform. The clutch statistic explained here tells us nothing about the quality of the player, and therefore cannot be use to evalute them properly. That’s why I think there is something innately flawed with this metric. I still think you should compare a players high leverage WPA to the league average high leverage WPA. It would then be possible to tell if the player is clutch relative to other players, which is more important than being clutch relative to one’s self.
I hope this makes sense, it was all one jumbled thought in my head.
Great point, Chris. I prefer stats that compare to a baseline like league average or replacement player, like OPS+, BRAA, etc.
Eric, I’ve been seeing your writing here and elsewhere and really enjoy it. Great work. Am I confusing you for someone else, or were you working on a book?
Chris, you make a good point for a different statistic. This was more to shed light on the statistic used here and how it really depends on our definition or perception of clutch. Nobody ever said we need to use it as a distinct measuring device. I usually just use it to see which players raise their games in these high leverage situations. To me, that’s clutch. But, for me as well, I’m not looking to make decisions based on clutch. I find it interesting that Juan Pierre, as I detailed the other day, has raised his game more than most others when in situations of crucial importance. I wouldn’t, however, use the stat to say that Pierre actually has value or is better than people think.
The way this stat works doesn’t require a baseline because each player is his own baseline. If you’re looking to see who performs the best in crucial situations that is much different than raising your game. The problem is that both have been tossed around as definitions of clutch. Some think it refers to players who perform well in crucial situations; others think it refers to those who raise their game with the game on the line.
It’s all in what you’re looking for.
Mitch, haha, yeah that’s me. There’s an interview with me up at The Hardball Times from yesterday where I discuss it in depth. It’s a book for those intimidated by statistical analysis wherein I present stats and history in an easy to follow analysis format, with the ultimate goal in mind that readers will not only understand advanced analyses but be able to conduct their own afterwards.
The problem I see with comparing each player’s “clutch” performance to a league average baseline is that not all clutch performances are created equally, and they’re not distributed equally amongst players (which pLI captures pretty well). Runner on third, one out, tie game in the seventh can be a pretty crucial moment, but is dwarfed in comparison to bases loaded, two outs in the ninth down by one.
In that case, you’d have to use an arbitrary cut-off point between clutch and non-clutch situations or group the situations by leverage index, et cetera, and by that point it’s getting so messy that why not just use Clutch in the first place? The beauty is that it captures not just the crucial moments (and weights them appropriately), but also the completely meaningless moments that Aaron Heilman, for example, so excelled at last season (0.96 WPA/LI, -0.71 WPA). So yeah. I never know how to end these things.
In the first sentence I meant to say “not all clutch *situations* are created equally”, not “not all clutch *performances* are created equally.”
Also, to Chris, I’m not sure Clutch was ever meant to evaluate ability. Instead, it seeks to boil down the contribution above or below one’s context-neutral performance into one tidy number, and I believe it accomplishes that goal quite nicely.
Nathan, you hit the whatever you hit on the head, on the head with that. The problem as I see it is that we have become a fandom that has mistaken clutch ability with plain ability. When did clutch become the gold standard for evaluating quality? The Clutch statistic here does a great job of evaluating how a player performed relative to a context-neutral setting. Did he step up? Stay the same? Digress? It’s interesting to know but not necessarily to base decisions on.
Like I said, I find it interesting that Juan Pierre has a higher clutch score since 2000 than David Ortiz, Albert Pujols, or Derek Jeter combined, but of course I would take all three of them over Pierre.
Eric and Nathan pretty much covered everthing here, but I’ll also note that over the long haul you can see whether or not a player is clutch, as in after many years of stats.
Clutch scores are fun and I think half of their point is to show that calling so and so “clutch” is rather pointless and some of players who you think of as clutch may not be.
I’ve also added this post to the references in the glossary (which I’ll get around to finishing soon hopefully).
Does your measure of “clutchness” correct for the quality of pitching faced in the high leverage situations? For example, a great hitter will usually face the opposing team’s best available reliever who has a platoon advantage if it is a late-inning, high leverage PA.
I’d like to propose an alternate definition of clutch. Clutch is the ability to maintain one’s level of performance in all situations, both high leverage and low leverage. Clutch would therefore be characterized more by a lack of dropoff in crucial situations than by an increase in performance in crucial situations.
Why? Because performing (significantly) better in high leverage situations means you perform (significantly) worse in low leverage situations, and why would that be a positive trait? Do we want to extol the virtues of a player who only performs at the top of his game when the game is on the line, but underperforms the rest of the time? I don’t. I want to praise the players who achieve the same high results no matter the situation, by maintaining focus in the low-lev spots, and mastering the pressure of the high-lev spots.
OK, now poke some holes in this viewpoint…
What Gator92 said. If there really are players who are consistently better in clutch situations, what that says to me is they have a failure of concentration when it “doesn’t matter”. Give me the players who are in the game all the time: because it almost always matters. The guy who starts the two-out rally down four runs is as important as the guy who finishes the rally, but clutch statistics (and our conception of clutch) doesn’t say that.
I think you explain well what your statistic is doing and what it isn’t. The only thing I would object to is the phrase “raising your game”, and that only because of the inference that most people will make that everyone can raise their game but only some do, while the truth might be that many players’ baseline game is their maximum game, and arguably everyone’s should be — at least in baseball (more so than in, say, basketball), made up as it largely is of discrete moments that the player ought to be able to focus on.
Gator92, all good points, though like I said to Chris, it’s a different type of measure. The point I was making with the post is that our definition of clutch is warped due to the vast amount of usage found in the media and how, quite frankly, no concrete definition exists. If we poll 10 people we’re going to get 7-10 different definitions.
The thing is that nobody ever said we had to rank players by clutch ability. If given three players, one with a 1.000 OPS in high LI, one with a .850 OPS in high LI, and one with a .700 OPS in high LI, I’ll take the 1.000 no matter if it constitutes a raising of the game.
Also, not sure what the purpose of the last line in your comment was, as I’m definitively against poking holes for the sake of poking holes but rather trying to clear up confusion and therefore the sarcasm is not warranted, in my eyes. Unless I’m reading it the wrong way.
Scraps, it’s really all in how you plan to use it. You could come up with something else that measures what you and Gator are discussing and it would be great, but it would be based on a different definition of clutch and therefore probably more likely to measure quality in high LI as opposed to just who posts better numbers in high LI than they individually do in all others.
Essentially, to try and sum everything up again, the Clutch statistic kept here is based off of the concept that clutch refers to a raising of the individual performance level in high leverage situations as compared to context-neutral settings.
It does not tell us who performs the best in high LI situations. That would be something different. A player with a 1.000 OPS in high LI performed better than someone with a .900 OPS in high LI, no questions asked; however, the definition of clutch varies from person to person so much and so often, that confusion arises from the stat kept here because many people mistake clutch for quality.
I admit it can be very confusing. Seeing Pat Burrell’s clutch score and realizing it ranks 3rd or 4th or wherever it is now in the entire sport makes us think that Burrell is the 3rd or 4th best player in clutch situations. He may be, but that is taking a statistic making measurement A and coming to conclusion B. He would have raised his performance in these situations the 3rd or 4th most but it does not tell us about the performance in general.
It may be not be a perfect stat but it definitely measures what it intends to measure. I personally feel this is what the concept clutch refers to. Others may disagree, and rightly so, but to me raising your performance level in crucial situaitons is clutch.
Then again, I will never use clutch to make decisions on a given player since it is more of a secondary or interest-inducing stat than a concrete one like WPA or WPA/LI. If I want to know the best performers in ‘clutch’ situations I’ll find who has the best slash line or OPS or baseruns or something else in those situations, since the overall numbers would be more indicative of the level or performance; the stat here indicates who performs better in them than they would normally.
I want to make it clear that I totally understand what you’re measuring. I just go even further than you do when you say you would never use clutch to make decisions on players: I think that if anything, it’s measuring a negative trait. The (theoretical) players with consistently high clutch scores are only raising their game because they’ve left room to raise it; the best players are playing their best all the time (or nearly).
(All of this assumes the existence of players who over time and large sample sizes consistently play better in clutch situations than normal situations. So far as I know — correct me if I’m wrong — such players haven’t been demonstrated to exist. And I know that there’s a big difference between clutch performance — what you are [usefully] measuring — and clutch players. I don’t mean to be criticizing what you are doing here at all.)
Eric, the last line wasn’t sarcastic, it was actually an acknowledgment (a playful one, at that) that this is a very wide-open topic, and that there are bound to be lots of alternative, valid ways to look at it. Some topics have clear right & wrong ways to look at it, but this is emphatically not one of those.
In fact, I’ll admit my post was a bit of a thread-hijack, as what you were looking at was the specific trait of elevating performance in high-LI, while I riffed off that and pointed out that it’s not necessarily a good trait…
You rightly pointed out that the clutch stat wouldn’t be a good one to guide a decision of who you would want at bat or on the mound in a critical spot – I agree completely, I view the clutch stat as you’ve outlined it to be interesting, but not likely to guide very many game decisions. Which is fine, of course, there are loads of stats quoted all the that are partly or completely like that… It is interesting to see some unexpected names on the good side of this list.
Guys, yeah, I get what you’re saying and I think the points you mention are EXTREMELY valid. I just wanted to write a post to clear up confusion on this particular metric.
I do agree it’s a stat that measures which players make up for bad performance elsewhere with good performance in clutch situations.
I actually didn’t create the stat so I won’t take offense to criticisms of it; I just want it to be known more publicly what exactly it tries to measure. It does a great job at measuring what it intends do, though it seems what it intends to measure might not jive with what everyone wants.
I think as long as we understand it’s a secondary type metric we’re all the better, but yeah, it’s definitely a wide open topic. I have a whole chapter in my book on clutch and stat-padding, about 25-32 pages, and I still don’t think I tackled even 75% of it.
Correct me if I’m wrong, but there a perception that good players don’t have high Clutch, because they are so good that they have no where to go but down or the same in clutch situations?
Do we know this is true? Do players with, lets say, above average WPA/LI, or whatever stat you want to use, have higher clutch scores than players with below average WPA/LI? Are there major differences in the top 10% and bottom 10% percentiles?
I might take a look at this when I get the chance if someone doesn’t beat me to it, but it seems to me that this is the predominant complaint?
Yeah, that seems to be the common perception and cause of “disdain” for the stat: a guy like Barry Bonds will post numbers better than anyone else in the league but because he performed worse than his insanely productive self in high LI than others, he’ll have a negative clutch score; however, someone like Miguel Cairo might have a .600 OPS but translate it to a positive clutch score because it becomes .700 in high LI situations.
Definitely an interesting idea.
Very helpful information, it has confirmed to me what I’ve suspected about Rick Ankiel’s ability to perform in pressure situations this year.
Now, is the end result here equivalent to wins? That’s the impression I have.
hi
j2vusm1ew6nrf1tc
good luck
hi
j2vusm1ew6nrf1tc
good luck