The Most Important Thing About Clutch
Last week, I wrote about the Dodgers. Not too long before that, I wrote about the Mariners. Both teams were and are still out of the playoff picture, but they’ve gotten to where they are in opposite ways. The Mariners are 74-57, but they have a BaseRuns estimated record of 64-67. The Dodgers are 70-61, but they have a BaseRuns estimated record of 78-53. The Mariners are four games better than the Dodgers in the actual standings. In the alternate standings, the gap is 14 games in the other direction. What on Earth must have happened over these five or so months? The Mariners have been clutch. The Dodgers have not.
We have our own clutch metric, as you know, that’s rooted in win probability. It’s called Clutch, and you can read a little about it here. In the interest of moving this post forward, here is an up-to-the-minute Clutch score landscape for all of MLB:
The Mariners are in first by a mile. The Dodgers are bringing up the rear. They’re not the only clutch and unclutch teams, but they’re the most *extreme* clutch and unclutch teams. The difference here is about 20 wins, based on timing alone. Timing has allowed the Mariners to look pretty strong. Timing has also caused the Dodgers to look surprisingly vulnerable.
Whenever I write about clutch performance, some of the same questions come up. The big one: Is clutch performance real? Now, for those of you who have been around for a while, nothing to follow is going to surprise you. I’ve written about this in the past, and my results today don’t look any different. But I thought this would be a good time to display all the data again. Clutch performances do happen. Obviously, clutch performances do happen, and they can add up over time. But, historically, it all just seems so random. You should never count on a team to be clutch.
We have win-probability data going back to 1974, so, we have Clutch scores going back to 1974, as well. There’s Clutch for team batting, there’s Clutch for team pitching, and if you combine them, there’s Clutch for teams overall. It stands to reason, I think, that if Clutch is a real skill, we should see it hold up over time. So, does it hold up over time? We can look at batting Clutch first, looking at teams in consecutive years:
There’s next to nothing there. If there were absolutely no relationship, we’d expect an R2 of 0.00. Instead, we get 0.01, with a best-fit line that has a very, very slightly positive slope. If you wanted to be extremely generous, you could say that batting Clutch has a slight tendency to repeat. Truthfully, though, this is a scatter. There’s no meaningful signal to be observed. What if we look at the same plot, but for pitching Clutch instead?
More of the same. Slightly positive best-fit line, yet a near-zero correlation. There’s nothing to bank on; there’s not really anything predictive. Unsurprisingly, this is what you get when you look at the combined total Clutch scores:
Clutch teams in one year haven’t gone on to remain clutch teams the next. Unclutch teams in one year haven’t gone on to remain unclutch teams the next. Obviously, some teams have repeated, but others also haven’t, which is what you’d expect if this were perfectly random. Half the time, a clutch team in year one would be expected to again be a clutch team in year two. That’s just how probability works.
Above, we looked at how Clutch holds up between seasons. There’s nothing significant, no strong pattern to be found. But what about the argument that certain individual seasons, certain individual clubhouses, just have something special going on? What if we looked at Clutch within seasons, splitting the first and second halves? I got all that information with the help of FanGraphs wizard Sean Dolinar. Here’s a plot of first- and second-half batting Clutch:
Nothing. No relationship. Clutch lineups early haven’t gone on to be clutch lineups late. The best-fit line is essentially flat. A team like this year’s Mariners is an anomaly. Here’s a plot of first- and second-half pitching Clutch:
Nothing, again. There’s just…look, there aren’t that many different ways for me to say “nope.” You know a random scatter plot when you see one. So just to finish it out, here’s a plot of first- and second-half total Clutch:
Nope. Nothing predictive. You can play like a clutch team, and if you do so, you’ll squeeze out some extra wins, but just because a team has been clutch doesn’t mean that team will *stay* clutch. The most clutch first half on record belongs to the 2016 Rangers, at +10.0. In the second half, they wound up at +2.8. Good! In second place there are the 2008 Angels, and they were also significantly clutch in both halves. But in third and fourth are the 1986 Red Sox and 1979 Astros, respectively. They both had below-average second-half Clutch scores. The least clutch first half belongs to the 1998 Mariners, at -10.0. In the second half, they wound up at +0.8.
I told you earlier you wouldn’t see anything new. If you’ve ever read much about clutch performance, it always seems to end up in the same place. But while I’m not blazing any new trails here, it does feel worthwhile to go over this all again every few years, just for the purpose of issuing reminders. This year, the Mariners have been clutch, and the Dodgers have not. Both of those statements are true, and their own players are the ones responsible for the timing of their own performances. But as far as the last month is concerned, or as far as next season is concerned, it’s practically irrelevant. There’s no historical reason to believe it’s a trait. It’s all just a collection of things that have happened. Things at important times that have wound up deciding ballgames.
This isn’t an entirely settled area. Some traits, like contact hitting, seem like they might allow a lineup to be a little more clutch. And we’ve seen some seemingly sustainably clutch teams in the past, like the recent Orioles, or the recent Royals, or the less recent Angels. You can always dig deeper, and maybe, in isolated instances, there really can be something special taking place. But if you’re not sure, if you’re not absolutely certain, history strongly suggests what you ought to believe. You ought to believe in the talent, more than you ever ought to believe in the timing.
Jeff made Lookout Landing a thing, but he does not still write there about the Mariners. He does write here, sometimes about the Mariners, but usually not.
Can you add a predictor variable of projected RP WAR? That’s always given as a reason for high clutch values.
Or scatterplot RP Clutch for half1 / half2?
Love it. This issue is the baseball community’s “negative results are still science.”
And just re “we’ve seen some seemingly sustainably clutch teams in the past, like the recent Orioles, or the recent Royals, or the less recent Angels” — has it in fact been demonstrated that there are more of them than would be expected to occur by chance? If in fact being a “clutch team” is a coin toss, you’d expect that there’d be a few teams turning up heads three or four years in a row, right? This might be a good subject for a mini-research project given that even the more statistically literate commentators have sometimes failed to exercise skepticism after a few years in a row of seemingly clutch performance — there were for instance some FanGraphs pieces about those Royals teams a couple years ago that really indulged the idea that their clutchness was nonrandom.
In my belief, most players are neither clutch or unclutch.
Therefore, it would be hard to argue that teams are clutch or unclutch.
However, I do believe in the existence of a few legit clutch/unclutch hitters.
Let’s look at Ichiro’s first 10 years in MLB.
His three worst clutch scores are -0.07, -0.02, and 0.48.
The average league clutch score during the decade per 678 PA is -0.08.
Ichiro was above average in clutch every single season.
I don’t think this is ‘a thousand people flip 10 coins and one of them will get all heads’ kind of thing.
So this suggests that a good speed+good bat control might lead to positive clutch.
And I thought of Dee Gordon, the poor man’s Ichiro.
Voila! Dee Gordon’s worst clutch score is 0.25 for the last eight seasons.
Is this any kind of proof? No.
But I do believe it is worth exploring.
It does seem like almost everybody agrees that the real underlying skills most likely to contribute to “clutch” outcomes are contact hitting and strikeout (i.e. contact-prevention) relief pitching. But it’s awfully hard to build complete teams that consistently possess a lot more than the rest of baseball of those skills, and even so there’s a ton of variance that’s more parsimoniously accounted for by a combination of chance and the more general set of “good at baseball” skills.
But … it has been. The Quest for Clutch is one of the original sabermetric holy grails. Even for players who are demonstrably clutch for a number of years, there’s never anything measurable about them that makes them stand out from other players of similar underlying skill. For all outward appearances, they are just the guys who got lucky and flipped “heads” 10 times in a row.
… which would make them roughly 1 out of a thousand.
Maybe clutch. Maybe just random noise as expected.
Looking at the same data used in the article- in two consecutive years: 50.8% were above or below average both years; we’d expect 50%, so we’re talking one extra of such teams for every two sets of two years.
For three years, being always clutch or unclutch was 27% with an expected 25%. That would suggest roughly one extra of such teams for every two sets of three years examined.
For four years I get 14.1% vs an expected 12.5%; an extra for every two sets of four years examined.
For five years I get 7.1% with an expected 6.3%; an extra for every *four* sets of five years examined.
For six years I get 3.6% with an expected 3.1%; an extra for every *seven* sets of six years examined.
For seven years I get 2.0% with an expected 1.6%; an extra for every *eight* sets of seven years examined.
For eight years I get 1.2% with an expected 0.8%; an extra team for every eight sets of eight years examined.
—
Now some notes.
* There were always more than expected I think we’d expect sometimes more and sometimes less, right?
* The expected number of extra stayed pretty stable and then fell off- assuming that clutch/unclutch** is more about the players than team philosophies, that might make sense as in the FA era there is only so long you could reasonably keep a team together
* I didn’t explicitly note it, but my impression while doing this was that the clutch strings generally out-numbered the unclutch ones; could it be that clutch teams are more likely to exceed expectations and be less likely to be blown up in future years and vice versa? I still have all of my data, so I may do a second pass splitting up clutch/unclutch runs.
** had a typo on unclutch and funny that Chrome’s suggested fix was ‘unlucky’ 🙂
Looking at just the clutch strings:
Two years: 24.8% vs 25.0% — almost exactly what would be expected
Three years: 13.5% vs 12.5% — 8% more than expected
Four years: 7.7% vs 6.3% — 22% more
Five years: 4.0% vs 3.1% — +29%
Six years: 2.0% vs 1.6% — +25%
Seven years: 1.4% vs 0.8% — +72%
Eight years: 0.9% vs 0.4% — +129%
Nine years: 0.6% vs 0.2% — +216%
Ten years: 0.3% vs 0.1%– +227%
Granted, now that I’m down this far one thing involved is sort-of double-counting.
The three teams in that last sample are :
2003-2012 Giants
2004-2013 Giants
2005-2014 Giants
IDK, is that OK or does it have to be adjusted for? That they went on a 12 year run, is it OK that this particular run counts 11 times in the two-year run bucket? And 10 times in the three year bucket?
From 2003-2014, the Giants were 28th in HRs, 20th in BB%, and 26th in K% (meaning 25 teams struck out more than them). Also, Barry Bonds, who clear doesn’t fit this mold, was with the team until 2007 and had a career Clutch score of -9.15. This goes along with the idea that high Clutch scores are associated with free-swinging slap hitters, which several people have mentioned here. This doesn’t necessarily mean that they are good clutch hitters in the classic sense, but the way that Clutch is calculated, this type of hitter does well. And other than Bonds, it’s clear that the Giants tried to get this type of hitter, that clearly was not a coincidence.
I wondered about individual players as well, interesting anecdotes. I also wonder if unclutch is more of a “thing” than clutch for individual players. Not necessarily a diminishing of skills/talent in high leverage situations. but it seems plausible that maybe some players could try to do too much or have a worse approach in high leverage situations that could lead to them executing or performing worse in those situations. More of a mental or emotional issue. So maybe not a skill or something even very repeatable, but possibly a “thing” for certain players?
Nick Swisher. The bigger the circumstances, the more he would overswing, thereby slowing his bat down and striking out even more than usual.
You had to know someone was going to bring the receipts, right?
Nick Swisher’s OPS across leverage situations was basically identical, trending worse in lower leverage situations.
All of the B-ref “Clutch” splits are pretty even across the board for him. His K rate is slightly higher in High Leverage situations than otherwise (22.6% in HL vs. 21.8% career rate).
Hitting in the 7th inning of a game in April is not what I consider “clutch”. Regardless, I would point out his lowest monthly batting average and OPS were in Sept/Oct, and his lowest batting average by inning was in the 8th and 9th innings.
What I was thinking was that for his regular season career, Swish had a .799 OPS, 113 OPS+. However, in the postseason, (across 12 series in 7 different years) Swish had a .165/.277/.297 triple slash.
The deeper in the playoffs he went, the worse he performed. Career .200 in the ALDS, .141 in ALCS, and .133 in his sole WS. Using the eye test and watching him for years, he would swing harder and harder in the post season, causing his bat to slow down and strikeouts to pile up. In the regular season, Swish struck out in 20.6% of his plate appearances. In the postseason, 25.9% of his plate appearances.
My guess would be that the orioles are because of zach britton while the royals are because they didn’t walk or strike out much.
So… maybe 6 teams out of the last 250 team-seasons?
That doesn’t mean much in the grand scheme of predictability or replicability of clutch. Like you point out.
High Clutch scores seem to be associated with free-swinging slap hitters (see other comments made here about Ichiro and the 2003-14 Giants), and that was also true of those Royals teams. If you try calculating a lot of WPA scores by hand, you’ll see why. In low-leverage situations, the difference between a walk and a single, and the difference between a strikeout and some other kind of out, is generally less than it is in high-leverage situations. The WPA difference between a homer and some other hit which drives in a run is only likely to be really big in terms of WPA in the relatively rare case that it’s a multi-run homer which ties a game or puts a team ahead when it had previously been trailing. Try calculating a lot of WPA scores by hand and you’ll see what I mean.
I regard the premise that there is no clutch as the central organizing principle of sabermetrics . That’s what makes it possible to ignore context.
There may be players who seem to be clutch year after year–Berra comes to mind–but how does one distinguish clutch from not giving 100% in low leverage situations? In recent years, Pujols has driven in a lot of runs while hitting very poorly overall. Some of that is having Trout in front of him, but isn’t it also possible that at his age and condition he takes some AB off?
Yes logically I know that there is no such thing but my God watching the Dodgers has been so incredibly painful this year. 13 losses when leading or tied entering the 9th inning. At some point they’re going to have to hold a 1-2 run lead in the 8th or 9th inning and win a game in regulation, something they have not been able to do in weeks. Literally weeks.
this week will be just crazy. 2 games at Texas not what a team with a mediocre pen needs at all…. Then the 4 with Arizona right after that. And also they can’t really if the pen gets toasted do what they normally would do- farming someone or coming up with a bogus DL excuse- as that would start a 10 day count for that pitcher to return. Especially when we get to Thursday or Friday- that would be not only this week’s Arizona series but also next weekend in Coors.
But on Saturday the roster size can increase
That is true, but it’s also 2 games into the crucial 4 game set with Arizona. LA sure doesn’t want this weekend to end being down by 3 or more games(with 25 games to play).
Pain for you..joyful for a Giants fan like myself 🙂
You do know, don’t you? Giants fans would rather the Dodgers lose than that the Giants win. It’s ingrained in us from all those pre-2010 seasons, which most of us (10-12-14) still can’t quite grasp as actually having happened. But we slide back into that role all too easily and 2018 is shaping up nicely.
The intra-season one is the most interesting to me given that
(a) personnel turnover makes year-over-year comparisons trickier and
(b) precisely because it addresses the “they had something magical that year” narrative that is spun.
Unless of course it’s the Curse of the Home Run Derby that’s breaking up all that first half clutchness. (Clutchiness? Clutchification? Clutchability?)
Great article, Jeff. All i had to do, really, was glance at each of the figures–or any one of them. The point couldn’t be more obvious.
And the final sentence of the article is very quotable!
Clutch is calculated by WPA-WPA/LI IIRC. How different would this look if we did Runs-Baseruns or Wins minus Baserunwins/WAR?
The specific stats which i think *may* be correlated with clutch are (1) highest war/xfip of any one relief pitcher on a team may be correlated with high clutch team pitcher scores (this of course depends upon the manager using this pitcher in high-leverage situations and this pitcher having at least average luck), (2) low walk-to-hit ratio for a team’s hitters may be correlated with high clutch (because walks generally have a smaller effect relative to hits in high-leverage situations as opposed to low-leverage situations, and (3) low strikeout percentage for a team’s hitters may be correlated with low clutch, because strikeouts are more likely to be the same as other outs in low-leverage situations.
What I wonder about….you’ve established through a previous article that the Dodgers are one of the worst clutch teams in recent history. Memorably bad, even. But that’s through August 26th. Question is, being of the random nature this is, are they MORE likely to continue to be a bad clutch team starting tonight until season’s end? I’m guessing no. I’m guessing that the random nature of the situation means its predictive ability for the rest the season is nil.
How do you wonder that after reading this article? This article answers that question.
My bad, I see it now. I got hung up on season-to-season and overlooked that. Thanks for pointing it out.
I think the problem that some have is that folks think that Arizona is just going to fall apart. Well, if they play the next 32 games at the same pace as they have all year long they’re winning 90 games. For LA to beat that, they would need to go 21-10 the rest of the way, a 110 win pace. They almost need to take 3 of 4 from Arizona this weekend.
well this is thoroughly exhaustive and i’m completely convinced.
is there any evidence, though, that certain players (i.e. not lineups) exhibit persistent tendencies in clutchness?
I won’t question the randomness of the team concept but there are too many individuals who produce more, maybe just some more but enough to make a big difference, when the game is on the line to ignore. There are levels of pressure that must be included in the measuring of what clutch really is. Hitting a 12th inning HR in a must win Game 4 of an ALCS then hitting a 14th inning walkoff single the next night or hitting a first pitch grand-slam in the 8th inning of a crucial game in a different ALCS when your team is down by four runs is CLUTCH. I always called Mo Vaughn, Mr. 10-2, because if the score was lop-sided either way he would often pad his numbers with a HR but fan when it counted.
For his Red Sox career, David Ortiz has a 0.0 clutch rating (slightly negative including twins years). I am one of the biggest David Ortiz fans out there but only remembering those hits is pure confirmation bias as he was an equal opportunity amazing hitter.
Mo Vaughn actually fits the profile statisitically of someone I’d consider “unclutch”. All of his clutch splits are below his average (except 2 outs/RISP) and for his career was about 7% worse in High Leverage situations.
His split with team lead >4R is 20% higher than his average. Sometimes our observations bear out in the data. (His K rate was about a point lower in High Leverage situations, though.)
Psssst. The reason why it seems so random is because *it’s not measuring anything meaningful*. You can calculate a statistic for almost anything. You could have a metric that is number of hits divided by number of walks, for example. I’m not sure that means anything more than what “Clutch” means. It might even be more meaningful.
Statistics with multiple inputs (like ratios, but also Clutch) are particularly easy to abuse because you’ve got two numbers going on, so a high ratio can mean one of two things. And add into that the complexities of calculating WPA (which may not even make sense when calculated at the individual level), and you’ve got a real mess of a statistic.
There may actually be clutch hitting that exists, but if so this is the wrong, wrong, wrong way to go about measuring it.
So I’m a little curious as to how WPA doesn’t make sense being calculated on an individual level. I can see arguments about fielding/baserunning effects, but that would be more of a problem with what the metric measures in general as opposed to how it is calculated with regards to individuals. If we could capture fielding and baserunning effects of individual events, it still is being applied to a set of individuals (opposed the reductionist P vs. Batter approach).
Because in a game that was won 3-2, the run in the 1st inning counted as much (and was as important) as the run in the 9th. WPA does not do that.
https://www.fangraphs.com/library/misc/wpa/
Calculating out the win probability added makes a ton of sense when you’re trying to figure out who is going to win the game while the game is actually happening. But at the individual level, if you want to calculate how much a player contributed to the team winning a close game, it makes more sense to calculate it retrospectively when we know how important each run was.
I think that the most likely objection is that “clutch hitting happens more when the game is close in the late innings” and that does comport with our idea of what “clutch” is. But since it introduces error into explaining who helped win the game, it’s a pretty lousy way to do it.
I prefer to think of WPA like this:
http://www.insidethebook.com/ee/index.php/site/article/wpa_is_wpa_is_not/
Anytime I’m trying to think about or talk about Clutch I just go back to Kris Bryant:
2015: 5th highest Clutch score
2016: 4th lowest Clutch score
I think a lot of people don’t consider the fact that it’s a zero sum game. If a hitter is “clutch” in a situation then there is a pitcher who is equally “unclutch”. It’s different then, golf, where there are no variable external factors, such as other players.
Unfortunately, the Fangraphs Clutch metric does not successfully isolate clutch performance. Players or team for whom batting average constitutes a large proportion of their offensive value will tend to have higher “clutch” ratings, whether or not they actually perform better in high-leverage situations. If you look at the BA:wOBA ratio for teams, you will see that “clutch” Seattle ranks 4th, while “unclutch” L.A. ranks 29th. That’s not to say the Mariners haven’t performed better when it counts — I assume they have — but the clutch variable needs to be redefined if you want to truly isolate that factor.
Clutch is not perdictable in baseball. However it is perdictable in football and basketball, why?
My theory is batting limits a baseball team’s ability to steer the ball toward clutch players in clutch situations. Basically a basketball, football or soccer team can push the play to a specific offensive player in specific times. Baseball only lets this specific matching happen on the defensive (pitching) side. My next question is are there teams that predictably limit their openet’s ability to hit in the clutch? Basically does a good bullpen make the opposing team predictably unclutch