Who Got Lucky in the Outfield?
You might want to buckle up. This is an article about small sample sizes, so there’s a statistically significant chance that things are about to get rowdy. It was supposed to be an article about which outfielders are better or worse than you’d expect them to be based on their sprint speed.
Just for fun, here’s the chart I started with. I turned Outs Above Average into a rate stat I’ll call OAA/150. It’s a player’s OAA per 1,350 innings, or 150 games. (I tried several other metrics, dividing by chances instead of innings, and working with UZR and DRS range metrics. This worked best for my purposes.) The sample is 544 outfielder seasons in which the fielder had at least 50 chances on balls with a catch probability below 96% (hereafter known as starred chances). I’ve labeled the two players who stood out the most in either direction in 2022.
Daulton Varsho good, Andrew Vaughn catastrophically bad. No surprise there, but I love charts like this, thick at the bottom and thin toward the top. They show how many paths there are to each outcome. Speed is a big component of OAA; the correlation coefficient of the two is 0.54. But outfielders also need to get good jumps, make plays at the wall, and be able to run down balls in all directions. There are lots of different combinations of skills that can land you in the bottom or middle of the chart. To get to the top, you need to be good at all of them. You also need to be lucky.
We often talk about the variability of defensive metrics, but it’s possible we don’t spend enough time on why. Myles Straw led all outfielders with 371 putouts in 2022. Pull in catch probability figures, and we can see that just under 80% of those were certain outs. Straw saw 114 starred chances, fourth-most in the league, but those were just 27% of his 429 total chances.
Batters can make as many as 700 plate appearances in a year; 114 outfield chances is a much smaller sample, and not every chance is created equal. Here’s a quick refresher on Baseball Savant’s catch probability metrics:
| 5-Star | 4-Star | 3-Star | 2-Star | 1-Star | |
|---|---|---|---|---|---|
| Catch Probability | 0-25% | 26-50% | 51-75% | 76-90% | 91-95% |
Working with the list of 545 player seasons, I took the number of chances from each bucket and calculated the percentage of each player’s total starred chances. Here’s the breakdown of the average player, along with the standard deviation for each bucket:
| 5-Star | 4-Star | 3-Star | 2-Star | 1-Star | |
|---|---|---|---|---|---|
| Average | 23.0% | 11.7% | 17.7% | 22.9% | 24.7% |
| Standard Deviation | 5.0% | 3.8% | 4.6% | 5.0% | 5.5% |
There’s a fair amount of variability, so I wondered whether the type of chances a player gets has any effect on their OAA. Here’s the percentage of each bucket broken down by its correlation to OAA/150:
| 5-Star % | 4-Star % | 3-Star % | 2-Star % | 1-Star % | |
|---|---|---|---|---|---|
| R-Value | -.21 | -.03 | .25 | .07 | -.07 |
Most of these R-values are too small to draw any conclusions, but we can definitely see a pattern here: Better OAA is correlated with fewer 5-star chances and more 3-star chances.
At this point I was really intrigued, so I went looking for a better sample. I pulled OAA data that wasn’t split into separate seasons and set the minimum number of chances at 200. This yielded a sample of just 122 players, but those players averaged 354 chances. We’re trading sample size for better samples. Those players also averaged 1.3 OAA/150, so the sample is skewed slightly toward better defenders. With the bigger sample size, I expected the variation in type of chances to quiet down, along with the correlation to OAA/150.
| 5-Star | 4-Star | 3-Star | 2-Star | 1-Star | |
|---|---|---|---|---|---|
| Average | 23.1% | 11.9% | 17.6% | 22.6% | 24.8% |
| Standard Deviation | 2.8% | 1.8% | 2.5% | 2.4% | 2.9% |
| R-Value | -.49 | .01 | .50 | .22 | -.16 |
As expected, the longer samples did cut down on variation between the kinds of chances. But look at the correlation between the percentage of chances and OAA/150! These are shockingly high values. On a single-season basis, sprint speed’s correlation to OAA/150 is 0.54. According to this sample, that’s not much stronger than the correlation between OAA/150 and the percentage of 5-star and 3-star chances. Frankly, I find it hard to believe that batted ball luck (or the opposite of batted ball luck, I guess) could make such a big difference.
At this point I was whatever comes after really intrigued, so I went back to the larger sample of 544 outfielder seasons and looked at the percentage of 3-star opportunities. In the chart below, each subset of outfielders is split into thirds based on who had the most 3-star chances. Keep in mind, whether they converted those chances does not factor into which bucket they’re in. We’re just looking at what percentage of their starred chances were specifically 3-star chances.
| Most 3-Star Chances | Middle 3-Star Chances | Least 3-Star Chances | |
|---|---|---|---|
| All Outfielders | 4.6 | -.02 | -1.3 |
| Center Fielders | 9.5 | 5.7 | 4.3 |
| Corner Outfielders | -1.7 | -2.6 | -4.7 |
Players who get more 3-star opportunities definitely have better results. I also split the fielders into groups based on their speed and their overall OAA/150, but I can summarize the data without making you look at yet another chart. Since the best and fastest outfielders are racking up OAA, the name of the game is maximizing value. They do the most damage on 3- and 4-star chances. Players with below-average speed convert 53.1% of their 3-star chances. While that’s not nearly as many as the fastest group of players, it’s still enough to make it the only bucket that’s correlated with better OAA.
It turns out that even the fastest players don’t convert enough 5-star chances to make them worthwhile; those are always correlated with lower OAA/150, unless you’re Jose Siri, whom the data indicates is some sort of wizard. For slower players, 4-star chances are nearly as uncatchable as 5-star chances, so they have the same negative correlation.
The differences correlated with 3- and 5-star chances are small, but they do let us identify the players who were hurt or helped the most by the makeup of their chances. And no one got luckier this year than Gold Glove winner Mookie Betts. He saw fewer 5-star chances than we’d expect and nearly twice as many 3-star chances. Going solely by his sprint speed of 27.3 feet per second, we’d expect him to post an OAA around -1.8. But if we use a model that factors in his 3- and 5-star chances, we’d expect his OAA to be 3.3, much closer to his actual total of 4.
On the other hand, Jake McCarthy, Bryan Reynolds, and Max Kepler were among the unluckiest outfielders in the league. All three of them saw just four 3-star chances this year; no one who played enough to be in our sample had fewer. Kepler is one of the best right fielders in the game, and with better luck, he could have bettered his 12 OAA, and maybe even outpaced Kyle Tucker for the AL Gold Glove.
Before you get on with your day and leave correlation coefficients behind you, I should mention defensive positioning. Since higher OAA/150 is associated with a fewer 5-star chances, I wondered whether better defenders were simply positioning themselves better, resulting in fewer tough chances. I wouldn’t be shocked to learn that players who are good at reading the ball off the bat and getting good jumps are also better at anticipating where the ball will be hit. But I couldn’t find anything to support that idea. Someone with access to better positioning data might be able to look into it further. If they do find that positioning has an effect, then what I’ve been calling luck may well be the residue of design.
OAA is not a perfect metric, and defensive sample sizes will always be small, so putting up a better score is not necessarily the same thing as being a better outfielder. Players don’t have any control over where the other team hits the ball. Just like you can’t fault the Guardians for feasting on a weak AL Central, you can’t blame a fielder for the balls that happen to come their way. But noting the makeup of those chances does give us a chance to refine the picture that OAA paints.
Davy Andrews is a Brooklyn-based musician and a writer at FanGraphs. He can be found on Bluesky @davyandrewsdavy.bsky.social.
This is some great defensive analysis! Maybe with better positioning data and statcast numbers on each fielding chance, there could be a figure to represent fielder batted ball luck. We could look at it maybe in the same way as BABIP
So glad for this article! Puts numbers on vague concerns I’ve had with a clear and solid method. Thanks!
Excellent article, which only reaffirms my distaste for defensive metrics being anything beyond a binary good/bad. Between small samples and other factors such as positioning, I just don’t trust the granularity at all when using it as a measure of skill. But this was a great bit of insight into what you called the picture OAA paints.
So the greatest differentiation in defense comes at the 3 star level. It might be interesting to see how well route versus jump and speed correlates with three star catches and how that might be different once you get to four and five. Since 3 stars are 51 to 75 percent catchable, the median ability is actually worse than a 50% catch rate with the above average fielders catching more of the difficult catches than the percentages may indicate. Maybe this is due to teams electing to employ the outliers in certain skills, like Miles Straw, instead of a more balanced offensive and defensive player.
Yeah…I feel like 51-75% might be too broad a range? Maybe that bucket needs more subdividing? Is there a breakpoint where skilled OFs distinguish themselves before the 4 star level?
Tangentially related: “Andrew Vaughn, catastrophically bad” in the outfield is the reason that Jose Abreu is now an Astro and the fact that many of my fellow White Sox fans don’t understand the advantages of a 25 year old first baseman over a 36 year old first baseman just drives me insane.
But it should be easy to understand a fan’s affection and attachment to a long time player who has won an MVP and been an impact player on the field and a leader in the clubhouse. In my opinion, that’s what being a fan is all about. Your favorite players are not just fungible assets.
I was thinking about the last part: “If they do find that positioning has an effect, then what I’ve been calling luck may well be the residue of design.” It’s certainly hard to measure, but if you tell your RF to hug the corners, then that’s obviously going to make them less effective in certain ways, even before the batter see that and maybe aims to go the other way, etc. It does make one wonder how much these numbers are determined by the “team” vs the individual.
Alternatively, if you have a great CF, then maybe your LF/RF looks better because you can set them up with more 2-3 star chances since Jose Siri makes their 4 star attempts look like 1-2 star shots?
An area for future research!
OF positioning is not just player skill but also impacted by team data and positioning strategies, who else is in the OF, and pitchers executing the attack plan.
A defensive wiz in CF surrounded by plodders will not only get more chances but also get tougher chances because he’ll cover more ground and the be more challenged by the edge chances than if surrounded by average or good fielders.
Likewise, an OF behind a good pitching staff that consistently executes the game plan is going to be less challenged that a similar OF behind a pitcher that doesn’t.
Basically defense is to a large extent a team effort and teasing out a single player’s contribution needs to draw on a whole lot more data (some of it team proprietary) before it can be as accurate as offensive analysis.
Doesn’t mean that attempts can’t be useful but results need a salt shaker handy.
After reading this, I’m curious how much of a player’s year-to-year OAA variation can be attributed to variability in batted ball opportunities.
Loved this article & deeper examinations of defensive metrics in general.
Maybe an idea for a future article (something I’ve always wondered about anyway) is why OAA doesn’t zero out positionally?
Looking at league positional splits for DRS and UZR they more or less even out with narrow variance around zero, but OAA has wilder swings with some positions plus or minus a hundred or more runs league wide.
Wasn’t it here on Fangraphs that someone did an article not too long ago that showed the best outfielders weren’t the ones who got the best read (first step) on the ball? It was speed that was the deciding factor.
The answer to the question is Mickey Mantle, under the right field bleachers by the Yankee bullpen, about the third or fourth inning