WAR Update: Catcher Framing!
Update: An earlier bug that impacted updated pitcher WAR has now been resolved. The pitcher tables below have been updated to reflect that. Thanks to everyone who pointed out the issue!
I’m very pleased to announce that FanGraphs has finally added catcher framing data to the site, with full thanks to Jared Cross, who you may know as the co-creator of the Steamer projections. We’ve also incorporated catcher framing into WAR.
Including catcher framing in WAR has been a topic of internal debate at FanGraphs for the past half-decade. The problem has never been with the inclusion of framing numbers on the catcher side of things. That’s a fairly simple addition. The problem has always been what to do with the pitchers. For instance, the 2011 Brewers were some 40 runs above average in catcher framing. When you add those 40 runs to catchers, do you subtract 40 runs from pitchers? As it turns out, you do, but those runs are not attributed equally to each pitcher:
| Player | IP | Catcher Framing | Framing per 9 |
|---|---|---|---|
| Randy Wolf | 212.1 | -0.45 | -0.02 |
| Yovani Gallardo | 207.1 | 7.79 | 0.34 |
| Shaun Marcum | 200.2 | 7.47 | 0.34 |
| Zack Greinke | 171.2 | 5.95 | 0.31 |
| Chris Narveson | 161.2 | 5.12 | 0.29 |
While most of the pitchers on the 2011 Brewers benefited from Jonathan Lucroy’s otherworldly framing, Randy Wolf was stuck with George Kottaras most of the time. In this instance, the entire Brewers pitching staff, with the exception of Randy Wolf, was a little bit worse once catcher framing is taken into account than their previous, non-catcher framing inclusive WAR would indicate.
Exactly how do you add catcher framing to WAR you ask?
For catchers, you take the catcher framing runs above average, divide by the runs to wins converter, and add it to your existing WAR total.
WAR = (Batting + Base Running + Fielding + Catcher Framing + Replacement Level) / Runs to Wins
On the pitcher side, we adjust FIP by the catcher framing runs above average per 9 innings. If Zack Greinke’s 2011 FIP was 3.00, and he was helped to the extent of 0.31 framing runs per 9 innings, we now use 3.31 in the WAR calculation instead of the original 3.00 FIP. We also adjust the pitcher’s dynamic runs to wins converter. In Greinke’s case, this would increase his personal run environment and also increase the runs to wins converter.
WAR = (((League FIP – (FIP + Catcher Framing / 9)) / Dynamic Runs to Wins Converter + Replacement Level) * IP / 9) * Game Start Leverage / 2
The RA9-WAR calculation has been adjusted in the exact same way.
Let’s take a look at how the inclusion of catcher framing has changed things:
| Player | Catcher Framing | Old WAR | New WAR | Difference |
|---|---|---|---|---|
| Brian McCann | 181.9 | 30.4 | 49.2 | 18.8 |
| Russell Martin | 165.6 | 29.5 | 46.7 | 17.2 |
| Yadier Molina | 151.6 | 34.8 | 50.5 | 15.7 |
| Jose Molina | 140.4 | 0.6 | 15.2 | 14.6 |
| Jonathan Lucroy | 126.8 | 22.6 | 36.2 | 13.6 |
| Miguel Montero | 127.0 | 15.6 | 28.9 | 13.3 |
| Yasmani Grandal | 119.6 | 15.1 | 27.6 | 12.5 |
| Buster Posey | 118.0 | 38.7 | 51.1 | 12.4 |
| Tyler Flowers | 89.4 | 8.6 | 17.8 | 9.2 |
| David Ross | 80.7 | 10.0 | 18.3 | 8.4 |
| Ryan Hanigan | 79.2 | 8.8 | 17.1 | 8.3 |
| Martin Maldonado | 69.2 | 4.6 | 11.7 | 7.2 |
| Jeff Mathis | 69.1 | -1.1 | 6.0 | 7.1 |
| Chris Stewart | 66.2 | 2.9 | 10.0 | 7.1 |
| Mike Zunino | 49.5 | 7.7 | 13.0 | 5.3 |
| Hank Conger | 48.1 | 1.7 | 6.9 | 5.2 |
| Rene Rivera | 48.1 | 3.9 | 9.1 | 5.1 |
| Player | Catcher Framing | Old WAR | New WAR | Difference |
|---|---|---|---|---|
| Ryan Doumit | -156.6 | 5.7 | -10.4 | -16.1 |
| Gerald Laird | -109.1 | 4.0 | -7.2 | -11.2 |
| Nick Hundley | -90.7 | 11.3 | 1.9 | -9.4 |
| Chris Iannetta | -89.5 | 17.7 | 8.3 | -9.3 |
| Kurt Suzuki | -86.1 | 18.1 | 9.0 | -9.1 |
| Carlos Santana | -78.6 | 14.7 | 6.4 | -8.3 |
| Salvador Perez | -79.9 | 17.8 | 9.5 | -8.3 |
| A.J. Ellis | -77.1 | 8.2 | 0.1 | -8.1 |
| Carlos Ruiz | -68.9 | 21.2 | 14.0 | -7.3 |
| Dioner Navarro | -65.4 | 5.6 | -1.2 | -6.8 |
| Lou Marson | -57.6 | 2.5 | -3.5 | -6.0 |
| Welington Castillo | -52.1 | 13.2 | 7.6 | -5.6 |
| John Buck | -52.4 | 7.2 | 1.7 | -5.6 |
| John Jaso | -51.9 | 8.0 | 2.5 | -5.5 |
| Rob Johnson | -48.4 | -1.5 | -6.5 | -5.0 |
| Robinson Chirinos | -47.7 | 8.3 | 3.4 | -5.0 |
| Player | Season | Catcher Framing | Old WAR | New WAR | Difference |
|---|---|---|---|---|---|
| Jonathan Lucroy | 2011 | 42.4 | 1.4 | 5.9 | 4.5 |
| Brian McCann | 2008 | 37.5 | 5.1 | 8.9 | 3.7 |
| Brian McCann | 2011 | 34.1 | 3.8 | 7.4 | 3.6 |
| Jonathan Lucroy | 2013 | 31.8 | 3.4 | 6.8 | 3.4 |
| Jonathan Lucroy | 2010 | 32.4 | 0.6 | 4.0 | 3.4 |
| Jose Molina | 2008 | 32.1 | 0.4 | 3.6 | 3.2 |
| Tyler Flowers | 2017 | 31.9 | 2.4 | 5.6 | 3.2 |
| Brian McCann | 2009 | 31.6 | 3.7 | 6.9 | 3.2 |
| Jose Molina | 2012 | 27.1 | 0.8 | 3.6 | 2.8 |
| Buster Posey | 2012 | 27.0 | 7.5 | 10.4 | 2.8 |
| Yadier Molina | 2010 | 27.2 | 2.2 | 5.1 | 2.8 |
| Russell Martin | 2011 | 26.6 | 2.5 | 5.3 | 2.8 |
| Russell Martin | 2008 | 28.1 | 4.8 | 7.6 | 2.8 |
| Brian McCann | 2012 | 26.4 | 1.5 | 4.2 | 2.8 |
| Buster Posey | 2016 | 26.7 | 3.8 | 6.5 | 2.7 |
| Jonathan Lucroy | 2012 | 26.1 | 3.4 | 6.2 | 2.7 |
| Y Grandal | 2016 | 25.7 | 2.8 | 5.5 | 2.6 |
| Miguel Montero | 2014 | 23.8 | 1.1 | 3.7 | 2.6 |
| Hank Conger | 2014 | 22.9 | 0.3 | 2.8 | 2.5 |
| Mike Zunino | 2014 | 22.8 | 1.7 | 4.2 | 2.5 |
| Player | Season | Catcher Framing | Old WAR | New WAR | Difference |
|---|---|---|---|---|---|
| Ryan Doumit | 2008 | -57.8 | 2.9 | -2.8 | -5.8 |
| J Saltalamacchia | 2014 | -31.8 | 1.5 | -2.0 | -3.5 |
| Gerald Laird | 2009 | -32.3 | 1.6 | -1.6 | -3.2 |
| Carlos Santana | 2011 | -30.3 | 3.4 | 0.2 | -3.2 |
| Carlos Santana | 2012 | -27.6 | 3.0 | 0.1 | -2.9 |
| Chris Iannetta | 2008 | -26.6 | 3.1 | 0.5 | -2.7 |
| Jorge Posada | 2010 | -24.2 | 1.5 | -1.0 | -2.5 |
| Kurt Suzuki | 2014 | -22.8 | 1.9 | -0.6 | -2.5 |
| Ryan Doumit | 2009 | -24.6 | 0.6 | -1.9 | -2.5 |
| Chris Iannetta | 2013 | -22.8 | 1.9 | -0.5 | -2.5 |
| Dioner Navarro | 2014 | -22.0 | 2.0 | -0.4 | -2.4 |
| Gerald Laird | 2008 | -23.9 | 1.4 | -1.0 | -2.4 |
| Ryan Doumit | 2012 | -22.2 | 1.0 | -1.4 | -2.3 |
| Dioner Navarro | 2008 | -22.6 | 1.9 | -0.3 | -2.3 |
| Miguel Olivo | 2011 | -21.2 | 0.2 | -2.0 | -2.2 |
| Jonathan Lucroy | 2017 | -22.1 | 1.1 | -1.1 | -2.2 |
| Lou Marson | 2011 | -20.4 | 1.0 | -1.2 | -2.2 |
| Lou Marson | 2010 | -20.3 | 0.5 | -1.6 | -2.1 |
| Rob Johnson | 2009 | -20.8 | -0.1 | -2.2 | -2.1 |
| Dioner Navarro | 2016 | -20.2 | -0.2 | -2.3 | -2.1 |
| Wilin Rosario | 2012 | -19.5 | 1.2 | -0.8 | -2.0 |
| John Buck | 2010 | -19.1 | 2.8 | 0.8 | -2.0 |
| W Castillo | 2013 | -18.3 | 3.2 | 1.2 | -2.0 |
And the Pitchers, where the differences are considerably smaller:
| Player | Framing | Old War | New War | Difference |
|---|---|---|---|---|
| Felix Hernandez | -23.3 | 42.7 | 45.4 | 2.7 |
| Justin Masterson | -20.7 | 14.2 | 16.4 | 2.2 |
| Jason Vargas | -21.0 | 12.9 | 15.0 | 2.1 |
| Justin Verlander | -17.6 | 57.0 | 59.0 | 2.0 |
| Ricky Nolasco | -12.4 | 23.6 | 25.0 | 1.4 |
| Mike Pelfrey | -13.6 | 11.8 | 13.2 | 1.4 |
| Kevin Correia | -12.3 | 5.5 | 6.8 | 1.2 |
| Cole Hamels | -11.1 | 41.4 | 42.6 | 1.2 |
| Anibal Sanchez | -11.7 | 25.7 | 27.0 | 1.2 |
| Zach Duke | -12.4 | 8.3 | 9.5 | 1.2 |
| Ubaldo Jimenez | -10.8 | 26.6 | 27.8 | 1.1 |
| Ian Snell | -11.9 | 1.6 | 2.7 | 1.1 |
| Derek Holland | -10.5 | 13.2 | 14.3 | 1.1 |
| Danny Duffy | -10.2 | 11.7 | 12.8 | 1.1 |
| Luke Hochevar | -10.1 | 8.0 | 9.1 | 1.0 |
| Paul Maholm | -10.2 | 11.4 | 12.4 | 1.0 |
| Edwin Jackson | -10.1 | 16.1 | 17.2 | 1.0 |
| Jeff Karstens | -9.6 | 3.2 | 4.2 | 1.0 |
| Roberto Hernandez | -9.7 | 4.2 | 5.1 | 1.0 |
| Player | Framing | Old War | New War | Difference |
|---|---|---|---|---|
| Yovani Gallardo | 25.6 | 21.3 | 18.4 | -2.9 |
| Bronson Arroyo | 28.6 | 8.9 | 6.1 | -2.8 |
| Madison Bumgarner | 23.4 | 30.7 | 28.0 | -2.7 |
| Tim Hudson | 24.5 | 14.5 | 12.0 | -2.6 |
| Kyle Lohse | 21.7 | 14.9 | 12.6 | -2.3 |
| Adam Wainwright | 18.6 | 35.3 | 33.2 | -2.1 |
| Jair Jurrjens | 19.2 | 9.7 | 7.7 | -2.0 |
| Derek Lowe | 19.0 | 12.4 | 10.5 | -2.0 |
| Ryan Vogelsong | 18.4 | 5.8 | 3.9 | -1.9 |
| Tommy Hanson | 17.2 | 9.5 | 7.6 | -1.8 |
| Johnny Cueto | 16.9 | 29.5 | 27.7 | -1.8 |
| Marco Estrada | 16.6 | 13.3 | 11.6 | -1.7 |
| Matt Cain | 15.7 | 21.1 | 19.4 | -1.7 |
| Ian Kennedy | 14.7 | 16.3 | 14.6 | -1.6 |
| CC Sabathia | 14.7 | 40.3 | 38.7 | -1.6 |
| Zack Greinke | 13.8 | 50.7 | 49.1 | -1.6 |
Now you know everything there is to know about how we added catcher framing to WAR. Please note the following:
- Catcher Framing (abbreviated as FRM) is available on the leaderboards and player pages in the fielding sections.
- WAR has been updated with catcher framing data everywhere WAR is available on the site.
- Catcher Framing data is available in batter and pitcher sections of the leaderboard as a custom stat.
- Fielding (the WAR component) now includes Catcher Framing runs above average.
- Steamer projections and depth chart projections both include projected catcher framing for catchers and pitchers.
David Appelman is the creator of FanGraphs.
A lot of these differences are stark, though we can understand a little better why Jeff Mathis is always a hot commodity. Also, I found it a little amusing that Kevin Correia and Mike Pelfrey were in the “biggest losers” table, and one of their catchers in 2013 with the Twins was none other than Ryan Doumit (he only caught 43 games, but still, it’s a fun coincidence).
There’s something more than coincidence here. Paul Maholm, Ian Snell, and Zach Duke are also among the biggest losers, and they were the pitchers with the most innings for the Pirates in 2008, when Doumit had a -55 Def score. Jeff Karstens was also a Doumit-era Pirate pitcher. Correia was also a teammate of Doumit in 2011.
This seems weird–if Doumit is getting a huge downward WAR adjustment, and the pitchers are also getting a downward WAR adjustment, where is that WAR going?
The law of conservation of mass and WAR is at stake here.
The randomness is what is bothering me with this. How are Lucroy and McCann going from greatest single season framer to below average in just a couple seasons?
Just on speculation – I would say that Umpires do not like to be manipulated. I read something on Fangraphs I believe about how umpires have begun to squeeze catchers with great framing reputations. That doesn’t change the value they earned the years before, just means there might be a sweet spot of framing value before the umpire community pushes back.
To be fair, other metrics have identified the same thing with Lucroy. And catching is so hard on the body; it makes sense that if something about your body changes, the subtlety of framing may be lost.
The article a couple of years ago examining how low Lucroy was squatting before and after his framing metrics changed was really good and made sense. Those extra couple of inches made a significant difference.
Another guess: maybe framing isn’t as much of a repeatable skill as we’d like to think. In other words, the same way that one pitcher will lead the league in BABIP one year and be below average the next.
It’s also possible that catcher may be especially good at framing certain pitches (say, fastballs) but not others (say sliders). Or sliders in the zone are ok but they struggle with sliders low and away to lefties. As pitch mix changes year to year, framing value probably does also.
It also could be a repeatable skill, but physically demanding and subject to decline the same way bat speed is. In that way it would not be random, but not constant either.
It is similar with defense stats.
There is a lot of randomness along with actual skill.
If you looked at wrc+ on a monthly basis, you’d observe “best hitters” turning into pumpkins.
The problem is teasing the skill part out.
However, there is additional concern here because the volatility of defensive stats can be explained by “sample size”. Yet, framing opportunities seem like it should be a much larger size so it is not satisfactory.
Another point that has been brought up is as framing became the stat du jour, teams/catchers started actively working on it and were able to pick it up pretty quickly so the average increased. (There was a fangraphs article about this phenomenon)
Individually, these don’t seem sufficient but all of them combined might be able to explain it. Having said that, I will still look at framing with several pinches of salt.
Jeff wrote a piece on this a while back. As more and more teams become hip to framing, the framing skill of the average catcher rises. Lucroy and McCann could have still been the same talent-wise, but the rest of the league caught up.
formerly matt w, maybe I’m confused, but aren’t the pitchers you mention getting *upward* adjustments, not downward? I’m seeing them in the “Largest Pitcher WAR Increases” table, where their history as Doumit’s teammates makes perfect sense.
You are correct. formerly matt w is the one who is confused.
This is great. Quick question: If Steamer is updated with this, does this mean that catcher framing gets rolled into the projected standings as well?
And Dan, if you’re out there, do you have any plans to try and roll it into ZiPs?
Also, in case anyone is wondering, Yasmani Grandal now is projected for the second-highest catcher WAR behind Buster Posey (which is–no lie–expecting him to TRIPLE his defensive projection from last year, and that ain’t happening). Realmuto is third. Gary Sanchez is fourth. Grandal, however, is projected for about 60-70 fewer PAs than the other three by the Depth Charts manager (whoever that is now).
Then why did teams not want to pay for Grandal? Does fangraphs really know better than each of the 30 major league teams and their analytics departments?
Grandal’s framing stats have been public on other websites for ages .. his reputation as a fantastic framer isn’t new. None of this data is new!
The fear with Grandal is probably mostly that he wanted a lot of money, catchers are known to decline earlier than other players, and a lot of contenders who would really benefit already have good catching situations (hence the Dodgers not re-signing him, etc).
There are also a lot of negative rumors out there about the one thing about catchers we still can’t measure – game calling.
He also was absolutely horrible in the playoffs every single year with the Dodgers.
And he said publicly that he signed earlier in the offseason than he otherwise would have because he wanted to know where he was moving his family to. He almost certainly could have gotten a multi-year deal if he had waited into February or March.
Reportedly, the Mets offered him 4/60. He rejected it.
Not only are Depth Charts and Steamer projecting Posey to triple his Def from last year, they are projected his Def to be higher than it was in 3 of the last 4 seasons.
Usually the make a lot of sense, but I sometimes don’t understand at all how projection systems come up with certain figures.
OK, so then the updated WAR numbers were indeed included in the Catcher Positional Power Rankings article that also came out today. They must’ve purposely pushed it back for that very reason.
My favorite thing about this update: Posey’s 2012 is now in the books as a 10 win season. Wow.
This might be a stupid question, but is there something that makes a pitcher better at getting frame-able pitches? Do pitchers have an effect on framing or is it something that is really more catcher dependent than anything?
I don’t think there’s any evidence to support it one way or the other. But…
Logically, the answer has to be “yes”, right? It’s got to be easier to frame for a standard fastball/changeup/slider guy than a knuckleballer. I think the bigger question is whether this method (and all the other methods) can take that into account, and whether it’s big enough to matter (knuckleballs aside). These methods are potentially (maybe?) robust to this sort of difference across pitchers.
Yes, they do. I don’t know if the main driver is a pitcher’s arsenal or command or something else but some pitchers get more/fewer strikes and it’s reasonably consistent year-to-year. Thanks for the comments folks!
Here’s one example: https://blogs.fangraphs.com/livan-hernandez-who-made-his-own-strike-zone/
We would see that manifested (in an easy to witness way) with the worst called balls series that Jeff used to do. The pitch would be down the middle but the ump would call a ball. Sometimes it’s a catcher doing a lousy job. Most of the time though it looks like the pitcher really missed their spot or crossed their pitches. Catcher sits high and inside and the pitch goes down the middle. Not the catcher’s fault at all. I would expect (but have no statistical information to back it up) that pitchers who generally have better command and hit their spots make it a heck of a lot easier for their catchers to post good framing numbers.
Now that probably doesn’t explain ALL of it. Clearly when a catcher improves the performance of ALL their pitchers then the bulk of the success must go to the catcher. But I suspect those are the outliers.
*cough* Greg Maddux *cough*
I have to understand the calculations better, but framing is THAT much of a difference-maker in terms of WAR?
My first thought is that catchers are part of the outcome on nearly every single pitch thrown. Not a very analytical answer, but that’s how I’m trying to understand it.
Yeah, that is exactly what framing data show, the number of extra strikes is huge for top framers and that translates to quite a few runs. That is why the whole of MLB immediately reacted to framing research in the last 5 years.
Difference between 0-1 and 1-0 is massive. As is the difference between 1-1 and 0-2. Also 2-2 vs 3-1. And 2-1 vs 1-2.
Add up those differences over the course of a season’s worth of pitches and the final result should be sizeable.
Makes you wonder what impact this would’ve had on catcher WAR prior to 2008…
The difference between 3-3 and 4-2 may also be significant, too
.
I, for one, welcome our new robot umpire overlords
I think this is a huge takeaway. It looks like Catcher framing has been completely disproportionately valuable over the last ten years. Robot umps would change this immediately. Andrelton Simmons defensive value OVER THE LAST 2 SEASONS is equal to McCann’s 2008. We need to decide, and quickly, if we want stealing strikes to be the most important defensive skill in baseball. We’re already ten years too late.
Personally, I do want it to be the most important defensive skill in baseball. Framing is fascinating to me, especially the psychological interplay with the umpire. It also rewards batteries that communicate well and are familiar with each other, which I think is something that should be rewarded. Watching a good framer take over the strike zone and help his pitcher get outs is really cool.
My first thought as well. Framing only exists/has value because human umpires are bad at their jobs. And I’m a little annoyed at the magnitude of the effect. How the catcher catches it should have ZERO bearing on whether it was in the strike zone. Humans are the worst.
Not even bad (with the exception of Bucknor, Hernandez and West), just not perfect. An umpire that is 95% accurate is missing thousands (tens of thousands?) of pitches per year.
Per the umpiring stats on baseballprospectus.com, the umpires who were behind the plate the most last year were in the 10-12k pitch* neighborhood. At 95% accuracy, it puts them at being off by 500-600 pitches.
*If those were total pitches and not called pitches, then even the 500-600 figure would be probably around double actual.
I think a 95% balls/strikes umpire would be among the best umps in the league (if not the best). I don’t have a source handy but do recall recently seeing something like 88% at the Joe West end and a league average of just over 90% (91/92%).
I think the pitcher WAR increase/decrease tables might be off for pitchers – lots of guys who threw to great framers in the increasers (Braves alone have Hanson, Hudson, Jurrjens, and Lowe, while the Brewers and Giants have a bunch as well).
Just to add to this, the first table says that “Positive framing numbers for pitchers indicate a pitcher was helped by the catcher’s framing ability” but the pitcher increase table appears to be *adding* framing to prior WAR, instead of subtracting it.
I noticed this as well. I was surprised to see increases for Bumgarner and Wainwright.
Yeah, I see. I’m looking into this, should have an update shortly.
This has now been fixed, thanks for the heads up!
Thanks – super excited for the new data!
Fun Framing Fact:
David Ross was as a Braves backup catcher from 2009-2012. In that time, over 663 PA, he hit .269/.353/.463 (123 wRC+) with 24 HR and 94 RBI. Add in his great framing, and he contributed 10.9 WAR over those 663 plate appearances.
A related fun fact: in 2009, the Braves main 2 catchers (McCann and Ross)* combined for 10.3 fWAR in 702 PAs.
*Includes time spent at other positions
This might be an indication that it is actually the pitchers, rather than the catchers making the difference.
It is certainly both. Whether it’s 80/20 pitchers, 80/20 catchers, 50/50, etc. is completely uncertain.
If framing was more highly valued back then, this suggests that the Braves could have improved significantly by trading one of Ross or McCann for a full-time starter at a position where they had zero above average players.
Do we retroactively give McCann the 2008 NL MVP? Wow
I think giving catchers full credit for this is a bad idea. We know from recent history that this skill is not very stable. We also know that pitchers contribute, i.e. some pitchers are easy to “frame”, some are not. I’d also bet pitcher reputation has an impact on umpires.
Pitchers are getting a share of the credit. The idea behind mixed models is to figure out how much of the credit belongs to each player. I’d describe a 0.7 year to year correlation as pretty stable (for catchers with >20 games) at least at baseball stats go.
Yeah, there’s just so many variables that can have an impact: pitcher delivery, consistency/accuracy, reputation; catcher familiarity/synchronization with pitcher; catcher framing ability; pitcher/catcher influence on umpire; umpire familiarity with pitcher; umpire tendencies for balls and strikes, etc.. Loved the link Benjamin gave regarding Livan Hernandez. It would be interesting to see a study on Chris Sale with his long, lanky, almost side-armed delivery. I’m sure he must scare the bejesus out of LH’d batters that have never faced him. I remember when John Kruk faced Randy Johnson for the first time in an All-Star game. He was so spooked, he wanted that AB over with as fast as possible.
It’s a mixed model? I thought when the article says “When you add those 40 runs to catchers, do you subtract 40 runs from pitchers? As it turns out, you do”, it was saying that 100% of framing credit is going to catchers.
Maybe this part is saying it’s a mixed model: “We also adjust the pitcher’s dynamic runs to wins converter. In Greinke’s case, this would increase his personal run environment and also increase the runs to wins converter”? I didn’t really understand what that meant.
Seems odd that several pitchers for the Cardinals and Giants had increases in their WAR even though those teams’ catchers (Molina, Posey) had increases in their WAR too. If memory serves, Lohse pitched with non-Molina catchers fairly regularly, but Wainwright certainly didn’t. And I don’t think Bumgarner or Cain were begging off of Posey. Any ideas?
I’m almost as blown away by the changes tables as I am by the Trout extension, which is saying a lot. Thanks for making this big change.
Besides the things people are pointing out about pitchers gaining WAR while pitching to great framers and losing WAR while pitching to bad framers, the new (negative) career WAR totals for Ryan Doumit and Gerald Laird aren’t the same as the new career WAR totals listed in the table above.
They started catching before 2008
Ah OK, that’s WAR from 2008 on. Which also means Doumit’s actual WAR total is surely lower than his current total. Yeesh.
Is it right that the higher the Catcher Framing in this formula the high the WAR?
WAR = (((League FIP – (FIP + Catcher Framing / 9)) / Dynamic Runs to Wins Converter + Replacement Level) * IP / 9) * Game Start
Should this really be part of WAR? It’s the players ability to manipulate the flaws of having a human umpire, not their ability to perform in the game like the ability to run, throw or hit.
It feels like it should be a calculated value and one that is very meaningful when evaluating the player’s contribution to his team, but it shouldn’t be considered when comparing how the player ranks historically against players at other positions.
If you feel differently, please explain it to me. I’m willing to listen.
I agree with you. We don’t have framing data before 2008 (at least, it’s not on the site). Which means there are drastic differences in WAR for catchers before 2008 that we aren’t accounting for. So, the entire notion of “best catcher all-time by WAR” has now gone the way of the dodo.
No, it’s still and will long be, Johnny Bench.
FWIW- the data used for modern fielding value on this site doesn’t exist for all-time. 2003 onward uses UZR and before that it uses TZ.
If you’ve been happily comparing pre and post 2003 players by fWAR despite the latter getting more accurate fielding value, then I’m not sure this change for before and after 2008 should be a huge deal.
I think the idea behind WAR is to measure a player’s on-field impact to his team. I believe this is done by calculated the net number of runs that he contributes relative to a “replacement”, and then converting these net runs to wins using some ratio (i.e. 4 runs above replacement = 1 win or something like that). What these writers have found is that the manner in which a catcher receives the ball from the pitcher can impact the number of runs that his team/his pitcher gives up. These framing metrics are an attempt to translate the catcher’s effect into the number of runs that he has saved. If we are comfortable saying that he saved/added X runs, I don’t see any reason we shouldn’t add that total to his other contributions from hitting, running, and other fielding effects.
Hopefully this makes some sense, and also hopefully it’s accurate enough to be meaningful. I definitely welcome corrections if my characterization of WAR is inaccurate.
Although catcher framing ideally *shouldn’t* affect ball-strike calls, it demonstrably does. Might as well give that credit to the catcher rather than his pitchers.
And if I’m a GM deciding which catcher to sign, I want the guy who gets my team more called strikes. Until robot umpires arrive, I think the new WAR is better than the old WAR.
Maybe in a vacuum, that’s true. But how do you suss out a player who hits well partially because his teammates steal signs? Or a base stealer who is consistently hiding tags? Or a pitcher (cough Pettite cough) who was balking on every throw to first?
I think it’s all well and good to say those things shouldn’t be considered a positive value, but there is no way to delineate them.
How is framing not an “ability to perform in the game”?
Framing is a physical act that is performed multiple times every game. It’s also a skill that has a 0.7 year to year correlation for individual catchers.
What’s the difference between rSZ and FRM on catcher defensive pages?
I understand that rSZ is from SIS but their values seem to vary pretty wildly from FRM.
What does rSZ stand for? It doesn’t show up when I hover over it or when I click it.
Did Ryan Doumit spend the vast majority of 2008 sneezing?
I’m a little confused by the results. How is it all the Pirates pitchers from the Ryan Doumit era ( Duke, Snell, Karstens, Maholm, Correia) are getting a WAR penalty for being caught by the worst pitch-framer with data available? Shouldn’t they be getting a WAR boost since Doumit is eating so much negative WAR that would otherwise have been theirs?
(Similarly, when I look at the leaderboards, Pirates pitchers from the Martin-Cervelli era are getting WAR increases from framing. Shouldn’t it be the other way around since they were caught by such excellent framers?)
Thank you David, consolidated a lot of interesting information just in this one article.
My first instinct is incredulity when I see the supposed magnitude of the effect.
We are meant to believe that Brian McCann and Russel Martin are now Hall of Famers?!
If a catcher can really have a massive percentage of their value in framing, what are we meant to do with all the catchers that played before these data were available?
In the end, is there really any good evidence that the catchers are responsible for umpires blowing calls? We know umpires are more likely to blow calls depending on the count no matter who is catching (e.g. extra strikes on 3-0, extra balls on 0-2).
…maybe the new market inefficiency will be teams selling high on catchers who are massively overvalued due to favourable framing numbers.
These framing numbers aren’t new and have been around for a while. Based on Grandal contract, I think your last paragraph is unlikely to come to fruition.
“If a catcher can really have a massive percentage of their value in framing, what are we meant to do with all the catchers that played before these data were available?”
Nothing, because we don’t have the data.
“In the end, is there really any good evidence that the catchers are responsible for umpires blowing calls?”
Yes. As Jared Cross mentioned in another comment, there is a 0.7 year to year correlation (for catchers with >20 games), which is pretty stable as far as baseball statistics go.
I haven’t followed BP’s discussions of this much, but has anyone found a reasonable historical proxy so something can be applied to past players? Defensive stats are done this way for other positions to some degree, and it would be very interesting to see all-time WAR rankings shuffled around some. Historical framing data would end up “flatter”, most likely, than the pitch-tracking version, but sheesh, these are huge effects.
The WAR rankings of the fangraphs top-15 catchers of all-time this morning just got a few new members and Ivan Rodriguez gained a bit to bump past Carter to 2nd all-time based on his last 4 years of framing (gained 6.5 runs, I assume he was behind Carter before). Since all the modern guys moved up their WAR based on this (Posey, Martin, McCann, Molina, Mauer, Rodriguez), it’s probably a decent guess that long-term catchers will typically do well by this measure, but there are always exceptions.
Yep. This really needs addressing. Otherwise we legit have no clue who the good catchers were prior to 2008.
I am also very interested in this. I’m sure it’s been attempted, which makes me worried that it’s not possible, since we haven’t heard any results in this area. On the other hand, this data doesn’t really matter to the teams, so we probably haven’t seen a lot of money poured into researching it.
https://www.baseballprospectus.com/news/article/20596/the-stats-go-marching-in-catcher-framing-before-pitchfx/
That piece gives a way to handle this from 1988 on, but that still leaves out a huge chunk of baseball history.
I think the post implies this, but has there been any modification to catcher WAR pre-2008, and is there any intent to do this in the future based on expected strike %?
Brian McCann is almost cracking the top 10 WAR among catchers now, well deserved!
I’m confused. How does Luke Hochevar have a decrease in WAR while the vast majority of his innings were caught by Salvador Perez and John Buck who also had large WAR decreases?
Same with Danny Duffy and Salvy
Is 13 FRM supposed to be read as 13 runs saved due to framing?
Seeing Jonathan Lucroy on both lists–man, what a fall.
Wow. And thank you.
Russell Martin and Brian McCann are now borderline HoFers? C’mon…
I miss Hank Conger and his ridiculous sleight of hand tricks. I imagine he’s working Vegas now.
He played in the Mexican Baseball League last year!
There seems to be something changing around 2013ish. Some of the consistently outsized framing numbers by guys like McCann, Molina, and Martin get cut by half or more in subsequent seasons. Is this an inflection point where more teams started actively working to exploit pitch framing, cutting into the good framers’ competitive advantage? And of course there is this. (https://blogs.fangraphs.com/pitch-framing-data-is-going-insane/)
I sort of feel like the volatile early numbers are going to prove to be a blip in the data. Probably even the guys who were good at framing didn’t know exactly what they were doing. It’ll make for some interesting/tedious hall of fame debates.
We don’t have framing data before 2008 (at least, it’s not on the site). Which means there are drastic differences in WAR for catchers before 2008 that we aren’t accounting for. So, the entire notion of “best catcher all-time by WAR” has gone the way of the dodo.
Unless there is some way to proxy this.
Well sure, if current fWAR was the end-all-be-all of stats, then you have a point. I’d just suggest thinking of the fWAR standings as “best players in order based on the available information”.
Modern players have more accurate defensive values thanks to UZR, while the pre-2003 seasons rely on Total Zone. UBR only goes back to 2002, so the baserunning components include much more stuff now than earlier in history when we basically had SB/CS only. And go back 70+ years and CS can be spotty or completely missing.
That fWAR has pretty much always included stuff for recent players that it couldn’t for older players and that has always been acceptable, to me means, that this change should probably still be OK.
If you believe that Brian McCann’s framing skill was worth two seasons of Mike Trout I’ve got shares in the Brooklyn Bridge for sale.
To be honest, these adjustments seem completely absurd.
How does this impact comparison across eras? For example, Jose Molina got a well-deserved WAR boost based on framing, which puts him roughly on par with Brad Ausmus. However, framing data from earlier eras is not avaialble, and some inferential studies have shown that Ausmus (among others) had phenomenal framing ability, which is presumably not accounted for in this new WAR.
Framing is the only measured skill with the exclusive goal of deceiving the umpire.
Dave, Jared, or anyone else — do you have any comments on the idea that valuing framing alongside everything else creates a discontinuity in the granularity of event that the system chooses to value?
Up until this change, every value-generating or value-debiting event in WAR came as a result of change in base-out state, i.e., a double, an event valued in FIP, making a catch, missing a catch, stealing a base, etc. Now, we have all those, plus extra value applied to catchers associated with changing the probability of a different base-out state event occurring. Does this give you any pause?
If probability-altering, non-event outcome things are to be valued, should we eventually consider moving to a second-order player valuation where we value pitcher ability to get into good counts, as well as hitter ability to work favorable ones? If the answer is “no” and this is unnecessary, why are catchers unique in this way to gain a non-event-based value?
This is a very good point. Pitchers are penalised the same whether they give up a home run on an 0-2 count or a 3-1 count. If extra strikes and balls are worth something to catchers, why not to pitchers? For pitchers they only become real value once the plate appearance ends.
Aren’t pitchers already rewarded for holding runners on?
I know this is true for the BP mixed model stats (namely DRA, possibly cFIP as well). Not sure it’s true at all in fWAR, and I can’t find documentation to that effect.
Catchers do get credit for preventing steals against them, even when no steals are attempted, in the same way that defensive arm scores for outfielders similarly give credit for no attempted advance rather than looking only at successful advance vs. guy thrown out. But in my view, those are not comparable, because the difference in “no advance” vs. “advance” is still the comparison of two base-out states, whereas there is no change in base-out states between “stole a strike that did not result in a strikeout” and “did not steal a strike [that would not have resulted in a strikeout anyway].”
Someday we’ll have an infinite-order Taylor polynomial analogue to spit out WAR values. For now welcome to the quadratic age
So Brian McCann now enters HOF discussions, with 30 WAR over a 4 year span from 2008-2011 and 56 WAR overall. Wow. And if pre 2008 is still unadjusted maybe that career WAR number should be a little higher too.
One might facetiously say he entered HOF discussions with 19 WAR during spring training of 2019.
Would it be possible to get 50/50 WAR for pitchers added to the customizable section?
Jose Molina now has a similar career WAR to Mike Napoli and Scioscia was right all along
I count 5 Angels catchers on the + list during the Scioscia era, and a swap of one of them (Mathis) for a – list (Iannetta) during the Dipoto era. This also coincides with the Angels decline.
He isn’t mentioned in the article, but peak Kershaw should be considered even better now since he threw to AJ Ellis.
Could you discuss what debates there were as to the *scale* of value added? According to your own leaderboards, Yadier Molina is now the second greatest defensive player of all time, with Russell Martin and Brian McCann in the top 10. Does … does this make any sense? Per game played, Brian McCann has had more positive defensive impact in his career than Brooks Robinson, Ozzie Smith, or Andruw Jones??
You seem to have the pitcher WAR impact backwards. The pitchers who gain the most WAR are guys who pitched to great framers, and vice-versa. Adam Wainwright, for example, should see his WAR *decline* — not increase — once you take account of Molina’s framing.
Yeah, there was an unfortunate bug, but it’s now been corrected.
How are both MadBum and Posey gaining WAR from this? Their careers overlap. If Posey is adding framing strikes, that means MadBum should be losing them.
Yasmani Grandal was the #1 Free Agent of the 2018 off season.
Can we add it to our custom dashboard? I didn’t see it as an option at a quick glance.
Unfortunately these changes make it impossible to do apples-to-apples comparisons, which is exactly what WAR was supposed to be, a tool that allowed you to compare players from different eras and different positions.
Since we only have framing stats from 2008-on, you can no longer compare catchers from before that era with those after. How should Gary Carter’s 1983 season (the best defensive catcher season of the last 40 years that doesn’t have a framing adjustment) compare with the 12 seasons ahead of him in the rankings, all of which are framing-boosted? How should Yadier Molina’s career compare with Ivan Rodriguez, when Molina’s version of fWAR includes a large bump from framing and Pudge has no adjustment up or down? Brian McCann passed dozens of catchers in career WAR as a result of this change. Some of those are certainly legitimate reshuffling. But has his career actually been better than Lance Parrish, a catcher ranked very highly for defense in his own era, who was ahead of McCann yesterday and now trails him?
How can you compare a pitcher from the framing era with those from a pre-framing era now? Fangraphs WAR calculation used to give an apples-to-apples comparison of how Kershaw compared with the legends of the past. Now Kershaw’s fWAR is calculated using a formula that includes framing adjustments, and those past legends do not.
I’m not suggesting we should ignore the greatness of Buster Posey’s 2012 season, which now ranks ahead of any season from Mike Trout (so far). But the entire purpose of having a stat like WAR was that it allowed you to make real comparisons across positions and across eras. And that is no longer true.
I agree, that it’s less of an apples to apples comparison than it used to be. Though, at least at a glance, it’s pretty easy to see which catchers and how much each catcher is impacted by Framing career wise:
https://www.fangraphs.com/leaders.aspx?pos=c&stats=bat&lg=all&qual=y&type=c,4,6,11,12,13,21,-1,34,35,40,41,-1,23,37,38,50,61,-1,111,-1,203,199,287,58&season=2018&month=0&season1=1871&ind=0&team=0&rost=0&age=0&filter=&players=0&sort=22,d
Pitchers on the other hand are impacted so mildly at the career level, that I don’t really see that as an issue. Though if you care to know:
https://www.fangraphs.com/leaders.aspx?pos=all&stats=pit&lg=all&qual=y&type=c,4,5,11,7,8,13,-1,36,37,40,43,44,48,51,-1,6,45,62,-1,300,59&season=2018&month=0&season1=1871&ind=0&team=0&rost=0&age=0&filter=&players=0&sort=20,d
Keep in mind FRM is in runs above average not wins.
Jorge Posada is an interesting case. He has positive defensive value for all the full years of his career prior to the framing years. Then his defense becomes all time bad. Even though he was still putting up HOF batting lines for a catcher, his fWAR now becomes negative for the end of his career with the inclusion of framing.
Anecdotally, early in his career many successful pitchers expressed their like for throwing to Posada (e.g. Clemens).
Wow.
First of all, thank you for providing this information. Huge day for Fangraphs.
Second, like most here I’m shocked by the implications. IF this is an accurate measurement of contribution, the impact can’t be overstated. An elite framer can double the defensive value of an elite SS or CF’er. I wasn’t anticipating that. A good framer can have as much defensive value (with framing alone) than an elite SS. This almost feels like a Moneyball moment. How could we (the collective baseball universe) have missed such a huge thing?
My gut tells me that the impact isn’t as large as what has been stated in these numbers, but I have nothing to back that up. I’m sure Jared and others have vetted the approach. I just need some time to digest and consider.
The magnitude in earlier years definitely takes some mental adjustment, but the magnitude of catcher framing numbers has been pretty consistent in study after study.
I’ve at times been in doubt in terms of how much the impact can really be, but pretty much everyone who has seriously looked at catcher framing has come up with numbers in a similar range, regardless of the method. The evidence has been way too consistent to ignore
i am simultaneously greatly fascinated and extremely uncomfortable with all of this.
Does team WAR correlate more closely to team wins after this addition?
Team wins will not have changed. We’re just shuffling where the wins go.
Is that fully right? I’m not sure it is, because wasn’t there some movement from catcher to catcher in addition to from catcher on same team to pitcher on same team?
That is, when Tyler Flowers’ WAR went up to 5+ wins in 2017, that wasn’t all the expense of his pitchers, right? Some of that was at the expense of other catchers who had below-average framing to offset his above-average framing, and those other catchers lost value, meaning that team wins re-shuffled, too.
Or am I mistaken? For what it’s worth, I looked at Devan Fink’s December 26, 2018 Beyond the Box Score article, where he used the now-legacy team wins/WAR wins data and found an r-squared of 0.86 between those two measures. I just re-did it using the current data and found a similar-but-marginally-different r-squared of 0.87. That suggests at least a little bit of re-shuffling, or else the value wouldn’t have changed. Or, again, did I screw something up?
If there’s any way to get the legacy data I’d love to confirm this either way!
Upon further review I think I’ve answered my own question that the re-shuffling is indeed intra-team rather than inter-team, so that’s neat. Before this adjustment and after, the Braves gained 0.2 on the position player side and lost 0.2 on the pitching side; the total of 40.7 stayed the same. If wins moved inter-team, then the Braves would probably be beneficiaries at the expense of another team due to Flowers, so I think that puts that to rest. Neat. (Would still love legacy data to confirm if available.)
Isn’t it likely that pitchers understand which catchers are better at framing, and adjust their strike zone accordingly? If Bumgarner is throwing the ball slightly outside the zone, knowing that Posey will be able to frame it for a strike, it’d seem that some of the credit should stay with the pitcher, rather than all of it going to his catcher. I think we’ve reached a world where we have to understand players’ intentions in order to fully capture the value of their contributions — meaning we’re somewhere beyond the possible.
Alright, time to discuss a new record-breaking extension for the real MVP, Fangraphs.
One more question that I’m sure has an obvious answer that I can’t figure out: on the leaderboards, if you aggregate by team and do framing by both pitchers and fielders, the FRM team value for fielders is always twice as large as the FRM team value for pitchers. If there’s a 1-to-1 reshuffling, shouldn’t they be the same? Why is one twice as large as the other?
Holy HOF Voters, Batman!
It’s going to be endlessly comical to watch the old guard wrestle with this, right after they were finally getting comfortable with WAR.
Posey and the M’s (Molina, McCann & Martin) just leapfrogged . . . basically everyone.
Using the career bWAR leaderboard as a stand-in, all four of those guys would have just bumped Thurman Munson from the 10th best career WAR for catchers to 14th!
I’m sure the math makes sense and all but I just have a really hard time wrapping my head around the idea that Lucroy’s pitch framing, and JUST his pitch framing, was worth more than all but ~30 players total contribution in 2011.
Great job FG! All 30 teams already do this internally, but great for us casual fans to see! Thank you.