What Is a Web Gem Worth?

The drama of a superlative catch at a crucial point in a game is one of baseball’s great narrative moments. A ball is struck and everyone – fans, baserunners, sprinting outfielders – holds their breath for a few seconds waiting for it to hit either leather or grass, sending those baserunners and swinging the game in one direction or the other. It’s baseball’s version of a three-pointer heading towards rim or net, or a wide receiver and a cornerback extending for the same airborne pass – a moment of suspense in the most literal sense of the term, during which the only thing drawing us closer to a conclusion is gravity.

Now, because there’s nothing baseball fans love more than taking a beautiful moment of athleticism, emotion, and aesthetics and distilling it into numbers, I’ve been mulling how to appropriately credit an outfielder for a play like this – particularly with respect to how it impacts the game in that moment. We have a pretty good measure for what a batted ball is worth if it falls in for a hit or is caught for an out, adding to one team’s chances of winning depending on the score and base-out situation – Win Probability Added. But what about when the ball’s in the air and it’s up to the outfielder to track it down? How much credit (or blame) is owed to the outfielder? How do we measure how much the outfielder’s defense itself swung the game?

Statcast’s catch probability can help us with our thought experiment. Consider a situation where the batting team has the bases loaded with two outs in the ninth inning of a tied game and hits a ball to the outfield with a 95% catch probability. That’s very likely to be converted into an out and end the rally — the defensive efforts by the outfielder aren’t likely to make much of a difference. But what about a ball hit with a 5% catch probability? At that very moment, ball in the air, the batting team would have a nearly 95% chance of winning the game. If both these balls are caught, the result is the same from a WPA standpoint, but the win probability chart will miss the fact that for one fleeting moment, it looked entirely like the batting team would win.

Using catch probability and win probability together, we can come up with a pretty good estimate of the value of a catch like this. It’s a similar idea to the approach used to convert Outs Above Average into runs prevented, where the run value of a completed defensive play is determined by calculating the difference between the expected run value of a batted ball and the actual value of the play made. To approach this question, we can look at the expected win probability value of a batted ball and compare that to its actual win probability value after the defender does or doesn’t make a play.

Let’s take a look at a real world example.

Last Monday, Eddie Rosario came to the plate to face Evan Phillips in the fifth inning with the Braves trailing the Dodgers by one. With one out and runners on second and third, it was a great scoring opportunity in a close game – with a run expectancy of 1.43 runs, the Braves actually had a 54.7% win expectancy entering the at-bat despite trailing. Rosario, who had homered earlier in the night, put a great swing on a 97 mph fastball on the outer edge and lined a hard-hit ball into center with a 96.3 mph exit velocity and a 14 degree launch angle.

According to Statcast, center fielder James Outman only had to travel 37 feet, but he had just 2.7 seconds to do it, giving the batted ball a catch probability of 5%. Outman broke in:

Now, let’s get one thing out of the way: it’s fine to debate whether the chances of Outman making this play were actually 5%. The probability of an outfielder making a catch on a given play is an exceptionally difficult thing to assess, and I don’t expect Statcast’s catch probability metrics to be infallible. That said, these are the most sophisticated metrics available to estimate the chances of an outfielder making a play, so while allowing for imperfection, I’m willing to trust it over my own hunches in most cases.

With that settled, we have a good sense of what recording an out here meant with respect to the outcome of the game. Atlanta’s win probability dropped 11.5 percentage points to 43.2%, giving the out a value of -.115 WPA (from the offense’s perspective). But that would have been the value of most outs as long as the resulting base-out situation didn’t change, like in the case of a sacrifice fly or outfield assist double play.

So the out in that spot was worth -.115 WPA, but how much did the catch swing his team’s chances of winning? What is the value of Outman converting a ball that otherwise would have at least tied the game into one that held the Dodgers’ lead?

To answer that question, we need to determine the expected value of the ball off of Rosario’s bat, which is not without its challenges. If we accept the premise that this ball would be caught 5% of the time, we can assume around a -.115 WPA in those cases when the run doesn’t score. The runner on third tags up around 40% the time batted balls like these to center field are caught, and according to Baseball Reference’s Stathead tool, sacrifice flies in this exact inning, base-out situation, and relative score situation have had an average WPA of about .000 over the last 20 seasons of data (which is to say, trading an out for a run in this particular scenario is neutral). So of batted balls like Rosario’s, we can roughly estimate that 3% will have about a -.115 WPA and 2% will have a .000 WPA.

But what about the other 95%? This ball looks like a pretty playable single if it falls in, and the vast majority of balls hit with that batted ball profile – especially right to the feet of the center fielder – end up as singles, so for simplicity’s sake, let’s say the alternative is a single. Yes, there are edge cases where Outman dives and the ball gets by him and goes for a triple, inside-the-parker or multi-base error, but seeing as something like that has happened around 1% of the time on similar batted balls to center field in the Statcast era, I think we’re ok to simplify here. Using the Stathead tool again, we can see that over the last 20 seasons, singles in this situation have driven in an average of 1.44 runs, giving the batting team a WPA boost of .133:

With a 95% chance of being worth .133 WPA, a 3% chance of being worth -.115 WPA, and a 2% of not moving the needle at all, Rosario’s batted ball could be estimated to have an impact of around .123 WPA in the Atlanta’s favor.

In other words, once Rosario made contact, the Braves could have expected to increase their chances of winning by 12.3 percentage points. Instead, they fell by 11.5. When Rosario started his swing, they had an estimated 54.7% chance of winning; when he hit it, that chance went up to around 67.0%; but when Outman caught the ball, it fell to 43.2%, a span of 23.8 percentage points covered within a three second period thanks to Outman’s glovework.

I want to be careful not to claim greater precision than this method is worth – we made a few assumptions and estimations in the process, and there’s some room for debate here in terms of how to best approach it. Feel free to quibble with any part of it, but the framework for this type of estimation is there. As we learn more about expected outcomes with tools like catch probability and batted ball metrics, we can better measure the impact of a player’s excellence in those moments.

When the catch is easier, the outfielder should get less credit for the swing in win expectancy. A few weeks ago in Baltimore, Jack Suwinski tracked down a Jorge Mateo fly ball with two outs in the sixth inning of a tie game and the go-ahead runner on third base. Suwinski traveled 73 feet, but had 4.5 seconds to do it, which Statcast says should be achievable 70% of the time. When the ball was caught, the out was worth -.061 WPA for the Orioles. Were it to fall in, as we would expect it to 30% of the time, it would most likely end up an RBI double – doubles in that scenario have been worth an average of .164 WPA over the last 20 years.

With about a 70% chance of being worth -.061 and about 30% shot of being worth .164, the expected WPA of the batted ball could be estimated at .007 – a much lower expected value than Rosario’s batted ball, of course, because it was much more likely to be caught. Suwinski caught it, good for a .068 swing – in other words, 6.8 percentage points of win expectancy – the Pirates’ way. That’s a nice impact, but appropriately less than Outman’s grab:

The same logic can be applied to non-catches, too. During the opening weekend of the season, Masataka Yoshida lofted a lazy fly ball into left field at Fenway, where Statcast’s catch probability says it had a 99% chance of becoming the final out of the game. Orioles left fielder Ryan McKenna fumbled it, though, leaving Yoshida on first and bringing the winning run to the plate in Adam Duvall:

This is a pretty straightforward one. The Red Sox had a 5.0% chance of winning when Yoshida came to the plate – according to Statcast, 99% of the outcomes would have dropped that down to zero, ending the game. In this instance, it more than doubled the Red Sox’ chances to 10.3%. Using our same logic, when the ball was in the air, it had an expected WPA value of -.049 (from the offense’s perspective), but when it landed on the grass instead of in McKenna’s glove, it had a positive WPA of .053, a swing of .102 WPA in Boston’s direction. Duvall would go on to hit a walk-off home run that McKenna couldn’t have done anything about.

Ok, sorry for picking on Ryan McKenna – here, watch his five-star catch from earlier in the same game:

A more complete analysis would break down the scenario further, determining the likelihood of all outcomes by a batted ball’s exit velocity, launch angle, and direction, predicting the WPA of those outcomes based on historical precedent, and then weighing those WPA values by likelihood to come up with the most accurate expected WPA possible – the less guesswork, the better. In any case, incorporating catch probability into the way we measure the value of a batted ball off the bat could open a lot of doors in terms of how we measure Win Probability Added. Instead of giving pitchers one-size-fits-all credit for the outs they induce, we could credit them based on how catchable a fly ball is, giving the pitcher more credit for a lazy fly ball than a gapper that his outfielder miraculously tracked down. And the same goes for hitters – we can add some nuance to a clutch hit by quantifying just how much a defender’s effort played a part. We could also assign leverage values to catches and measure not only which fielders added the most to their team’s success with their glovework, but which one did the most in proportion to the opportunities they had.

The bottom line: Win probability isn’t static from the point a pitch leaves a pitcher’s hand to the end of the play – in fact, it is often swinging moment by moment within the action of those plays themselves. With a closer look, we can give more shape to what’s happening between a pitcher’s delivery and the play’s conclusion. The better we get at projecting the probabilities of defensive plays being made and, thus, the expected values of batted balls, the closer we can get to measuring those game-altering moments of suspense in all of their volatility.