FanGraphs Pitch Framing

In 2008, when Dan Turkenkopf was the first to quantify the value of pitch framing, he noted that it appeared to be alarmingly important. Bill Letson was similarly astounded when he calculated the size of the effect in 2010. Max Marchi and Mike Fast each took a turn in 2011, finding large and highly correlated catcher values despite using different methods. Other sabermetric luminaries have contributed sophisticated methods and sanity checks, some of which I’ll touch on below. And yet, this terrifically important, seemingly well-established, and impressively repeatable defensive skill has been left out when calculating FanGraphs player WAR and ignored when Steamer forecasts pitchers and catchers. . . until now.

In what follows, I’ll briefly lay out a series of steps for calculating how many framing runs each catcher contributed as well as and how many extra strikes each pitcher was granted (or, in some cases, earned). This much has all been done and clearly described before thanks to Dan Brooks and Harry Pavlidis; that research was updated and expanded upon by Pavlidis and Jonathan Judge. I’ll then compare the values I’ve obtained to the ones created by Baseball Prospectus, StatCorner, and Sports Info Solutions and demonstrate (I hope) that those extra strikes really do result in extra strikeouts and fewer walks. Lastly, I’ll discuss what this means for Steamer forecasts.

Modeling the Strike Zone

It all starts with the strike zone and I started by using generalized additive models to estimate the probability of a strike in any count, to either left-handed or right-handed batters at each location in and around the plate. On the first pass (shown below), I created strike zones averaged across seasons and, on the second pass, looked for changes in the strike zone by season. The blue contour lines in the images below show where strike calls are a coin flip and the red dashed lines show where we’d expect a 25% or 75% chance of a strike. If you’re read Matt Carruth or Jon Roegele, you’ll be unsurprised to see a small 0-2 strike zones (shown in the upper right facets) and large 3-0 strike zones (in the lower left).

Distributing Extra Strikes and the Case of Ryan Doumit

We can now compare actual strike calls to predicted strike probabilities. I used a logistic mixed effects model (with batters and pitchers as random effects) to split up credit for extra strikes between pitchers, catchers and dumb luck (or, if you prefer, unattributed variance). For now, I left umpires out of the model reasoning that pitchers and catchers each see roughly their share of pitcher-and-hitter-friendly umpires. Also, I’m looking for a model that we can update daily throughout the season and adding random effects quickly adds computational time.

These extra strikes can be roughly translated into saved runs at a rate of 0.135 saved runs per additional called strike. When we do that we find the following highlights and low-lights of (PITCHf/x era) framing history:

“Negative 57.8 runs?” I hear you cry! Well, Jeff Sullivan asked this same question when confronted with Baseball Prospectus’ estimate of -63 runs for this same Doumit season in an article entitled “How Bad Could a Pitch Framer Possibly Be?” Sullivan noted that Doumit was estimated to have lost 200 runs due to poor framing over the course of his career. For a sanity check on this number, he compared how pitchers fared with Doumit behind the plate relative to other catchers on the same team and found that pitchers had in fact allowed 213 more runs with Doumit.

Comparisons to Baseball Prospectus, StatCorner, and Sports Info Solutions

The following charts compare our new Framing Runs to the ones provided by Baseball Prospectus, StatCorner and Sports Info Solutions. Each data point is one catcher season. I’ve kept the axes the same across all three graphs so that you can gauge the relative spreads in runs. One thing that jumps out is that Sports Info Solutions is decidedly more cautious in their numbers than the other three systems.

The following tables shows the correlations between the systems from 2010 (the first year of SIS numbers) through 2018 and the year-to-year correlations for catchers from one season to the next. Our model appears to be most similar to the Baseball Prospectus model but all four models are highly correlated. Our model and BP’s model have the highest year-to-year correlations for catchers but, again, all four systems are in the same ballpark.

xxFIP really works (and so does pitch framing)!

In 2013 Chris Carruthers asked “How can we take a good ERA Estimator like xFIP, and eliminate framing and bias?” To answer this question he created metrics with inputs that are not affected by the umpire call: swinging strike rates, ball in play rates, zone and out-of-zone looking rates (with zones defined by the rulebook rather than by the umpires) and foul rates. Carruthers created expected strikeout (xK%), walk (xBB%) and xFIP (xxFIP) metrics using only these bias-free inputs. I updated these metrics so that their means for each season match the means of pitchers’ actual rates and then asked whether the differences between pitchers’ actual strikeout rates, walk rates and xFIPs and Carruther’s metrics could be predicted by pitch framing.

More specifically, I built regression models to predict the differences between a pitcher’s actual rates and Carruthers’ bias-free metrics based on the pitcher’s framing runs per game (including runs attributed to the pitcher himself, to his catchers and to luck).

If all these saved strikes our model thinks it sees, really do cause a divergence between xFIP and Carruther’s xxFIP in just the way we might imagine, we’d expect every run per game of framing to correspond to 0.92 runs of difference between the biased and un-biased ERA estimators (since 92% of runs are earned).

According to our regression model, each framing run predicts a 0.89 difference (with a standard error of 0.02). Not bad! We might also be interested in how much framing affects strikeout and walk rates.

Our models show that each framing run (per game) adds 3.9% to a pitchers strikeout rate and subtracts 3.9% from his walk rate. Or put another way, it adds 7.8% to the difference. What does this amount to in terms of runs? For that, we can use Tango’s kwERA formula:

kwERA = 5.40 − 12 · (SO% − BB%)

Adding 7.8% to the difference between SO% and BB% should add 12 · 7.8% = 0.94 to a pitcher’s ERA. Again, this is consistent with the 0.92 we’d expect.

These regression models gives me confidence that, although the magnitude of our pitch framing numbers seems astounding, it’s not far wrong. Or, in truth, it gives me further confidence since I was already a believer. If you’re still on the fence, I’d recommend reading Mitchel Lichtman’s research which shows that catcher projected to be good/bad framers really do save/cost runs in the way we’d expect.

Player Projections

With framing numbers now in hand, it’s time to start forecasting. The following table shows a projection for Yasmani Grandal who is expected (by Steamer) to catch 120 games and see 7,599 called pitches (within a 3.5’ x 3.5’ box around the strike zone) in the upcoming season. The “Framing Rates” are Framing Runs per 6000 called pitches and the seasonal weights are how much weight we give to each called strike from that season. The projection is the weighted average of Grandal’s past framing rates with 1000 league average called pitches added to the mix (the “regression towards the mean”). The upshot is that Grandal is expected to save 17.5 runs by virtue of his framing in 2019. His teammates, Manny Pina and Erik Kratz, contribute an additional 1.3 runs of framing to bring the Brewers team total to 18.8 runs.

We use the following three steps to adjust pitcher projections based on framing:

1. Adjust past strikeout and walk rates to what they would have been with typical ball and strike calls. This includes removing any benefit pitchers received due to extra strikes regardless of whether these strikes were attributed to the catcher, the pitcher or plain old luck.

2. Forecast 2019 strikeout and walk rates (in the usual way) using these adjusted strikeout and walk rates.

3. Adjust these forecasted rates based on each pitcher’s projected ability to get extra strikes as well as the abilities of his team’s catchers.

The affects on individual pitcher’s projections are generally quite small. Pitchers who were good at getting extra strikes are projected to continue to get extra strikes going forward so that much is mostly a wash. Pitchers expected to benefit from catchers with soft hands often benefited from these same skilled framers before (meaning that good framing was baked into their numbers already) and, even where their battery mates have changed, they’re only feeling a fraction of that catcher’s contribution to the team.

Nonetheless, here are the pitchers who benefited the most from adding framing to Steamer.

. . . and the pitchers who took a hit.

We can add this all up and see how this affects the number of runs each team is expected to allow.

The Royals, Brewers and Rangers feel the benefit of acquiring Martin Maldonado, Yasmani Grandal, and Jeff Mathis, respectively, while the Mariners get dinged for adding Omar Narvaez and the Diamondbacks feel the loss of Mathis.

WAR Adjustments

Once we’ve estimated how many runs each catcher saved (or, in the case of projections, might be expected to save) we translate those runs into wins and adjust WAR accordingly. We can adjust pitcher WAR by doing essentially the inverse and subtracting out the value of catcher framing. Essentially, we’re shifting runs (and thus wins) between pitchers and catchers since catchers deserve some share of the credit (or blame) for what transpired.

Please see David Appelman’s WAR post for all the details.