Pitch-Framing Data Is Going Insane

The season’s complete, which means the numbers are official. This is convenient for a writer, because it means there shouldn’t be any issues anymore with comparing 2017 to another full season in the past. A full season is a full season. So how about a quick full-season review of the pitch-framing data? There’s something interesting going on. Something dramatic, something that shakes the foundation of the numbers themselves. I have the graphs to prove it.

The most advanced pitch-framing information is available at Baseball Prospectus. It’s long been the gold standard, and so many hundreds of hours have gone into generating the results that get published on the sortable leaderboards. There are two framing metrics of note, for catchers and for entire teams. One is just framing runs above average, which is self-explanatory. The other is CSAA, or called strikes above average. This is directly related to framing runs above average, but it’s expressed as a rate stat. I think that’s all you need to know. In this post, I’m going to use them both.

There exist 10 years of detailed information, based on the pitch-tracking technology that’s been in existence. This is a plot of standard deviations, year to year, over the course of the decade. This is on the team level, using runs above average. This is just an examination of each year’s spread.

There’s currently less spread than there was in the past. There hasn’t been any real change since 2015, but there’s still less spread than in, say, 2008. This is presumably related to a point I’ve made before: More teams than ever are aware of the value of good framing. So more teams are prioritizing it, which makes it harder to stand out. When you raise the floor, you narrow the distribution. There are still differences between the best teams and the worst, but the gaps are somewhat smaller. Neat stuff.

But that’s not really what I want to show you here. Instead, I’d like to call your attention to something taking place with individual catchers. I gathered data for every catcher since 2008 who’s had at least 2,000 framing opportunities in consecutive seasons. Here’s how the numbers held up, in terms of CSAA, between 2013 – 2016.

You see that pretty strong, linear relationship. You want that relationship to exist; that way you can have the confidence you’re measuring something real and sustainable. That plot comes with an R^2 value of 0.49. The slope is 0.69. A good framer in one year was likely to be a good framer in the next year, and the opposite was also true.

Moving on now, here’s the same plot, except for only 2016 – 2017.

Look, there’s still some relationship. This isn’t the picture of randomness. And yet, the R^2 value is 0.20. The slope is just 0.42, which is down 40% from in the earlier plot. Something just happened. Or, something is continuing to happen. I think this last plot drives it home. Here are all the year-to-year R^2 values, in terms of CSAA.

The sustainability is eroding. Which means the predictability is eroding. Sure, there was a little spike a year ago, but the last three years have the three weakest relationships in the sample. And, actually, the last four years have the four weakest relationships in the sample. Which pitch-framing performance was first being measured, one of the things that made it so exciting was that the numbers held up so well, year to year. It was essentially proof of signal. The method of measurement hasn’t meaningfully changed ever since. It’s the same system. If anything, it’s more advanced now than ever. It’s had the benefit of time. But the year-to-year relationships are disintegrating. A good framer in 2016 was still likely to look like a good framer in 2017, but that couldn’t be said with very much confidence. The data is getting increasingly random.

Welington Castillo is currently a free agent. He just spent the year with the Orioles. When he became an Oriole, he had the record of being a below-average framer. Last year, he performed like an above-average framer. Ditto J.T. Realmuto. Ditto Stephen Vogt. Buster Posey, meanwhile, got a lot worse, and so did, say, Tony Wolters. I have a sample of 393 catcher season pairs. Of the 24 biggest year-to-year changes in CSAA, seven of them just happened between 2016 and 2017. I don’t even know how to explain what’s happened with Chris Iannetta.

Of those 393 catcher season pairs, Iannetta is responsible for nine of them. But of the seven largest year-to-year changes, whether for better or worse, Iannetta’s been responsible for three of them. All three are from the most recent seasons. From 2014 to 2015, Iannetta got dramatically better. From 2015 to 2016, he got dramatically worse. And from 2016 to 2017, he got dramatically better again. Iannetta might be the current face of pitch-framing uncertainty. Or maybe it’s Jonathan Lucroy, who just keeps on declining. I don’t know. Things are just weird.

There are a few possible explanations. One, it’s all a blip. I don’t know. Maybe. Numbers do funny things sometimes. Two, there’s something wrong with the actual data, which might be related to the recent switch-over from PITCHf/x to Trackman/Statcast. That wouldn’t explain what already seemed like a trend before 2017. And all this information comes from Pitch Info, which takes deliberate care to make all necessary adjustments and corrections.

And three, get used to this. This could be the new normal, the consequence of more teams caring about how their catchers catch. Maybe framing is easier to teach and learn than we thought. Maybe more catchers than ever know what they’re supposed to do, and so the baseline for everyone is so high that random volatility plays a larger-than-ever role. Even if everyone were exactly the same, there would still be variation, because the baseball season isn’t infinitely long. If every team, for example, were a true-talent .500 ballclub, a season would still end up with 90-win teams and 90-loss teams. You could detect this volatility because, in subsequent years, you’d observe further randomness. Performances wouldn’t correlate so well year to year. That’s what we’re seeing with framers.

In a sense, you could see this all coming. It’s been possible to forecast, and I’ve written about this on multiple occasions. But it’s still pretty stark to look at that 2016 – 2017 R^2. A far weaker relationship than ever. Seemingly far more randomness than ever. Is Welington Castillo actually a good pitch-framer now? I never would’ve believed it, but the data says what it says. I don’t know what to think about Castillo, and I don’t know what to think about a lot of different guys. Pitch-framing has entered a strange new era. An era in which it still matters, but an era in which it’s not easy to tell who’ll actually stay good at it. It’s difficult to justify a heavy investment in something that now comes with such a high degree of uncertainty. But that same uncertainty is what the league has to reckon with.