Pitch Framing Park Factors
Back in March, we introduced catcher framing numbers on FanGraphs. Not long after, Tom Tango noted in a blog post that pitch framing numbers should be park-adjusted since pitchers and catchers in some parks are getting more strike calls (relative to Trackman’s recorded locations) than others.
We can see this in the graph above, which is based on called pitches within a 3.5 x 3.5 inch area in and around the strike zone. There are, on average, 64 pitches per game that meet this criteria so this graph essentially shows how many extra “framing” strikes pitches and catchers were assigned in each park per game. Put another way, this tells us how many more strike calls they received than we’d expect based on the recorded locations of the pitches. We’d certainly expect some spread in the results for home team pitchers and catchers, since some teams have better framers than others, but we shouldn’t see such a large spread for road pitchers and catchers, whom we’d expect to have essentially average framing talent. We also see that there’s a strong positive correlation between extra strikes for the home team and extra strikes for road team, suggesting that the park itself plays a role. There are two big outliers here — Sun Trust Park and Coors Field, both in 2017. Something must be amiss at those parks and we should control for it when calculating our framing numbers.
Adjusting Pitch Framing Numbers for Park Effects
Just as when constructing other park factors, we need to be careful to account for the quality of the players playing in each park. We’ll need to account not only for the pitchers and catchers who played in each park but also for the batters, some of whom have fewer strikes called against them. What we need is essentially a WOWY (with or without you) calculation where we find each park’s tendency to yield strikes, controlling for the pitcher, catcher, and batting team. In practice, it’s easiest to do this with the help of a mixed effects model. We can take the mixed-effects model we used to estimate pitcher and catcher framing and simply add random effects for the ballpark and batting team.
After adjusting for the park and batter effects that we find, we can take another look at the graph that led us here and compare home and road framing at each park, but this time with park-adjusted numbers.
This looks much better! With park effects removed, we still have a significant spread in home-team framing but a relatively small spread in road-team framing.
New Pitch Framing Numbers
For most catchers, our park adjustments make little difference. The graph below plots the new framing runs for catcher-seasons against the old framing runs with 2017 performances shown in red.
The tables below show the team-seasons, catcher-seasons, and catcher careers most affected by the park adjustments.
| Team | Season | Old FRM | New FRM | Park Bias |
|---|---|---|---|---|
| Rockies | 2017 | -26.2 | -9.6 | -16.6 |
| Rangers | 2017 | -25.8 | -12.2 | -13.6 |
| Blue Jays | 2010 | -0.5 | 10.9 | -11.4 |
| Mariners | 2017 | -8.2 | 3.0 | -11.2 |
| Tigers | 2017 | -24.1 | -13.1 | -11.0 |
| Team | Season | Old FRM | New FRM | Park Bias |
|---|---|---|---|---|
| Braves | 2017 | 29.3 | 9.4 | 19.9 |
| Orioles | 2017 | 13.2 | -0.4 | 13.6 |
| Braves | 2009 | 47.0 | 38.2 | 8.8 |
| Brewers | 2010 | 44.4 | 35.9 | 8.5 |
| Pirates | 2008 | -51.7 | -59.9 | 8.2 |
| Player | Season | Old FRM | New FRM | Park Bias |
|---|---|---|---|---|
| Jonathan Lucroy | 2017 | -22.1 | -10.1 | -12 |
| James McCann | 2017 | -16.2 | -8.1 | -8.1 |
| A.J. Pierzynski | 2010 | -5.8 | 2.2 | -8.0 |
| Mike Zunino | 2017 | 2.4 | 10.2 | -7.8 |
| John Buck | 2010 | -19.1 | -11.7 | -7.4 |
| Player | Season | Old FRM | New FRM | Park Bias |
|---|---|---|---|---|
| Tyler Flowers | 2017 | 31.9 | 20.5 | 11.4 |
| Austin Hedges | 2017 | 21.8 | 12.8 | 9.0 |
| Kurt Suzuki | 2017 | -2.9 | -10.9 | 8.0 |
| Welington Castillo | 2017 | 1.6 | -6.3 | 7.9 |
| Yadier Molina | 2017 | 8.7 | 1.8 | 6.9 |
| Player | Old FRM | New FRM | Park Bias |
|---|---|---|---|
| A.J. Pierzynski | -41.9 | -21 | -20.9 |
| A.J. Ellis | -77.0 | -59.9 | -17.1 |
| Joe Mauer | 13.7 | 27.5 | -13.8 |
| Jonathan Lucroy | 126.9 | 139.6 | -12.7 |
| Wilin Rosario | -39.5 | -29.3 | -10.2 |
| Player | Old FRM | New FRM | Park Bias |
|---|---|---|---|
| Brian McCann | 181.9 | 162.0 | 19.9 |
| Welington Castillo | -52.0 | -66.0 | 14.0 |
| Miguel Montero | 127.0 | 113.6 | 13.4 |
| Wilson Ramos | 21.2 | 8.3 | 12.9 |
| Ryan Doumit | -156.7 | -165.7 | 9.0 |
Jared Cross is a co-creator of Steamer Projections and consults for a Major League team. In real life, he teaches science and mathematics in Brooklyn.
lol at Ryan Doumit getting actually worse
Ryan Doumit’s 2008 (-3.4 fWAR) is literally unbelievable:
* 4th worst of all-time for a position player (no min. PA), and the worst following 1977.
* By far, the worst for a player with an above average wRC+ (123), followed by Doumit’s 2010 (-2.2 fWAR with a 102 wRC+). The next worst (2007 Jermaine Dye) was just -1.2 fWAR.
* The next worst season for a player with a wRC+ of 120 or higher was Brad Hawpe’s 2008 season (122 wRC+, -0.7 fWAR).
* The worst season by DEF ever, with the runner-up more than 15 runs better (Adam Dunn’s 2009, which resulted in 1 fWAR amazingly).
Truly unbelievable that one guy could be worth over 60 negative runs in a season on defense, especially in only 106 games. Makes me skeptical of the measures, even though I know he was a really bad C. His DRS was only -3 that year. Something tells me that we are still missing something with regards to framing values. At the same time, gotta feel bad for Maholm, Duke, and Snell.
Why do parks affect pitch framing? Different parks shouldn’t affect how the umpire sees the pitch, should they? Studies have shown that a major factor in home advantage is umpire bias. Maybe they are more biassed for the home team in certain parks than others. But that doesn’t seem to fit with the general finding that the bias affects both teams.
The tables showing the most and least park biasses don’t indicate any correlation with run scoring environment, as I expect they wouldn’t. So the question remains, what is it about the park that affects the umpires? Is it harder to see pitches in certain parks than others? But if that were the case, I’d expect a correlation with run-scoring environment.
I’m surprised that neither this article nor the Tango blog even attempts to speculate on a cause.
If the batter’s eye is known to impact the batter, it makes sense it would also impact the umpire, I did not regard it as a surprising idea when Tango posted it or when I saw this article today.
If there are K/BB park factors for batters (and there are), then of course they impact umpires as well.
But the greatest biasses as listed in the tables do not correlate with K or BB park factors.
And by the way, K and BB factors correlate negatively and positively, respectively, with run factors.
I wonder if it has more to do with the Statcast calibration at different stadiums.
Isn’t this just about instrument “error” and a correction for it? Like the idea is that the cameras at each park are calibrated in some way that can result in systematic bias towards what they perceive as a strike versus what an umpire will, and by inserting a correction for “park effect” you’re basically just correcting for the assessed static component of that instrument “error.”
Calibration’s also a factor, absolutely, I had not even thought about that one.
If it’s instrument error, then all the PF data have to be thrown out, don’t they?
Not if the error is consistent/systematic, then doing a few basic statistical tricks (like the ones applied here) would control for said error.
If it isn’t consistent/systematic, then yeah, that’s hard to wrestle with.
Yes, but the point is, if there is an error, than the PF data as they stand now, are inaccurate, and have to be changed.
And I still wonder why this wasn’t mentioned by either Tango or the author here. If the problem is calibration, then there really aren’t park factors for framing. There are park errors that mistakenly appear as park factors.
I’m not seeing much of a difference there!
In addition to batter’s eye or other visual conditions (shadows for example), I think part of it could be climate factors. For example, pitches don’t break as much in high altitude, and maybe straighter pitches are a little harder to frame (maybe these are easier for the umpire to judge, and therefore harder to frame). Temperature also affects the ball, and we can see that in flyball data, so that may affect pitches as well.
I think the effect of the batter’s eye is an interesting idea but I think that calibration is the more likely primary explanation since it’s not carrying over from one year to the next and the effects appear largest soon after a new system was put in place.
Coors and Sun Trust’s 2017 seasons are pointed out as outliers, but the data includes 11 seasons per stadium. I’d be curious where the rest of those seasons are on the first graph since there are no other data points near the outliers.
Can we revisit the fact that we’re counting pitch framing in WAR, but it’s the only element in WAR that doesn’t affect a game state (outs, runs, runners on base)?
Do we give credit to hitters that strike out, but fouled off a lot of pitchs before that? Or pitchers getting to 0-2, but then giving up a HR?
I’m not opposed to the idea of catcher framing, but something doesn’t sit right with me. Maybe it’s the magnitude? But then again that’s how a lot of people feel about Fielding metrics, and I think they’re out of touch and refuse to change with the times, so what does that make me?!
The issue is that framing has enormous cumulative impacts because we know the outcomes between 1-0 vs 0-1, or 2-0 vs 1-1 vs 0-2 etc are so significant in the long scale, and the catcher handles many, many more pitches than any individual batter sees.
With batters, we have concrete events to assign value to. If we had concrete events for catchers, that would be a better way, but we don’t: we credit the entire result to the pitcher. To tease out that portion of the catcher’s value away from the pitcher, we need to get into smaller chunks than batting requires, because there’s two parties involved instead of just one.
Just to pick an example, Cody Bellinger has seen 2158 pitches this season; he’s missed like 4 games.
Will Smith has already had 1,300 framing opportunities and he has caught 215 innings so far this season. (I’d use a regular catcher like Yasmani Grandal, but BP’s framing leaderboards that include that data are currently doing their morning load and the leaderboard is incomplete). Grandal has caught 850 innings this year, and there’s little reason to think that framing opportunities are super variable from staff to staff, so that would mean that Grandal has had something over 5,000 framing opportunities (which are just called pitches near the zone, not including swings etc), which is more than double Bellinger’s total number of pitches seen (which includes swings, and he has a 44% swing rate).
You don’t want to take the entire credit away from the pitcher for, say, a strikeout that is framed by the catcher, so you can’t just give the catcher credit for the K (cause you can’t give credit to both). Going down to pitch-level lets you take only some of that credit away from the pitcher and leave it distributed more equitably.
It would be better, if for no other reason than acceptance, if there were discrete events to tie the framing value to in the way you describe, but it doesn’t exist in a way that works into the rest of WAR’s framework, and we’re much better off having some measurement of this than not, because it is so pervasive and can impact so many plays per game. It’s an enormous cumulative benefit. If Jonathan Lucroy has provided 160 framing runs of value in his career, that seems like an impossible number, but its going to be spread out over like 60,000 opportunities.
A guy like Russell Martin or Brian McCann have probably actually had more than 100,000 framing opportunities in their careers.
Another way to think about this is that there are other things that happen in baseball as frequently as framing opportunities but they are all incredibly routine so there is almost no value differential from player to player in them. Routine ground balls and fly balls are caught at 98-99% rates by even the worst fielders (the differences are in the more marginal/difficult plays). There are thousands of those plays every year, but there’s no differentiation between players among them. We’re used to the things that differentiate players being things that happen a few times a game, not a dozen times an inning.
But framing differentiates players and can happen 10-20 times in a single inning. When you consider that kind of scale, the magnitude of the eventual results makes sense.
That’s some real Muddy Waters!
https://www.youtube.com/watch?v=bSfqNEvykv0
😉
I know this is little consolation to you as I am just one internet human, but I have the same uncomfortable feeling about it. I think if you go back to the original framing announcement post, I expressed the same concern. It is a thorny issue for sure. You can make a case for either inclusion or exclusion, but the reality is that the effects are massive, even if there’s a bit of a strange factor in finally, on this particular hill, giving guys credit for interim, reversible state changes where no such credit existed before. C’est la analytic decision-making vie.
Six of the top 10 park effects were in 2017, but occurred in ten different parks (Turner for 2009 ATL, Suncoast for 2017 ATL)? LuCroy gains 12.7 runs, with 12.0 coming in one season? Is there any year-to-year correlation for parks?
I understand you want to be more accurate, but you’re saying you had a 100% error on James McCann in 2017 based on a factor you can’t logically explain?
Conceptually, I understand and appreciate the value of framing.
Methodologically, framing has been and is now a complete mess. We need to show a bit more humility about our ability to glean insights from the mass of data we have.
Just one more reason to hope that robo umps arrive soon. The calibration problems illustrate why they aren’t here yet, but at some point they presumably will. Framing may be a skill, but conning the umpire shouldn’t have value. The idea is that players on one team try to out-perform players on the other team. The umpire should not be involved in this game.
There is a ~0.3 year-to-year correlation in park bias that goes up to ~0.4 if you exclude 2017.
Right, the idea is to be as accurate as possible. Is there a better alternative than estimating these effects as best as we can and then adjusting for them?
I’m not digging at the willingness to try and estimate the effects as best we can; I’m just unwilling (justifiably, I think) to put much stock in the accuracy of those efforts with the enormous operationalization problems we have here. I feel the same way about pitch values – they are entertaining, but there are entire analyses on this site based on them that have essentially no validity.
MGL did some work demonstrating that good framing catchers really do affect results to the extent that framing numbers suggest and I showed in March that framing does really correspond to differences between expected K% and BB% (based on swings, fools and takes at different pitch locations) and actual K% and BB%. There are high correlations between systems with different methodologies and between catchers year to year. Some skepticism is always warranted but I think you may be making the mistake of doubting too much.
Are Steamer projections accounting for these findings?
Yes.
If you are going to control for pitchers, catchers, and batters, why not control for umpires also?
A good question and the reason maybe isn’t so satisfying. The more random effects we add, the longer the model takes to fit (and right now we’re updating the numbers nightly) and, since each pitcher and catcher see a mix of umpires, the umpire effect evens out and we can get away with not including it.
This may be true, but it could also be very enlightening to look at framing stats by ump. I would guess you’d see a spread of results that could result in outcomes like managerial decisions such as which catchers play vs which umps.
I’m also gonna guess the Rays have already developed this 😉
Yeah, I like that idea!