Pitcher zStats Entering the Homestretch, Part 1 (Validation)

One of the strange things about projecting baseball players is that even results themselves are small samples. Full seasons result in specific numbers that have minimal predictive value, such as BABIP for pitchers. The predictive value isn’t literally zero — individual seasons form much of the basis of projections, whether math-y ones like ZiPS or simply our personal opinions on how good a player is — but we have to develop tools that improve our ability to explain some of these stats. It’s not enough to know that the number of home runs allowed by a pitcher is volatile; we need to know how and why pitchers allow homers beyond a general sense of pitching poorly or being Jordan Lyles.

Data like that which StatCast provides gives us the ability to get at what’s more elemental, such as exit velocities and launch angles and the like — things that are in themselves more predictive than their end products (the number of homers). StatCast has its own implementation of this kind of exercise in its various “x” stats. ZiPS uses slightly different models with a similar purpose, which I’ve dubbed zStats. (I’m going to make you guess what the z stands for!) The differences in the models can be significant. For example, when talking about grounders, balls hit directly toward the second base bag became singles 48.7% of the time from 2012 to ’19, with 51.0% outs and 0.2% doubles. But grounders hit 16 degrees to the “left” of the bag only became hits 10.6% of the time over the same stretch, and toward the second base side, it was 9.8%. ZiPS uses data like sprint speed when calculating hitter BABIP, because how fast a player is has an effect on BABIP and extra-base hits.

ZiPS doesn’t discard actual stats; the models all improve from knowing the actual numbers in addition to the zStats. You can read more on how zStats relate to actual stats here. For those curious about the r-squared values between zStats and real stats for the offensive components, it’s 0.59 for zBABIP, 0.86 for strikeouts, 0.83 for walks, and 0.78 for homers. Those relationships are what make these stats useful for predicting the future. If you can explain 78% of the variance in home run rate between hitters with no information about how many homers they actually hit, you’ve answered a lot of the riddle. All of these numbers correlate better than the actual numbers with future numbers, though a model that uses both zStats and actual ones, as the full model of ZiPS does, is superior to either by themselves.

And why is this important and not just number-spinning? Knowing that changes in walk rates, home run rates, and strikeout rates stabilized far quicker than other stats was an important step forward in player valuation. That’s something that’s useful whether you work for a front office, are a hardcore fan, want to make some fantasy league moves, or even just a regular fan who is rooting for your faves. If we improve our knowledge of the basic molecular structure of a walk or a strikeout, then we can find players who are improving or struggling even more quickly, and provide better answers on why a walk rate or a strikeout rate has changed. This is useful data for me in particular because I obviously do a lot of work with projections, but I’m hoping this type of information is interesting to readers beyond that.

As with any model, the proof of the pudding is in the eating, and there are always some people that question the value of data such as these. So for this run, I’m pitting zStats against the last two months and all new data that obviously could not have been used in the model without a time machine to see how the zStats did compared to reality. I’m not going to do a whole post for this every time, but this is something that, based on the feedback from the last post in June, people really wanted to see the results for.

I addressed hitters earlier this week. Today, we’ll turn our attention to the pitchers. Starting with zBABIP, let’s look at how the numbers have shaken out for the leaders and trailers from back in June. I didn’t include players who faced fewer than 100 batters over the last two months.

zHR was closer on 16 of 21 overachievers, with a mean absolute error (MAE) of 0.9% vs. 2.1% for the actual home run rate. This is kind of to be expected since zHR is going up against an extremely volatile stat — zHR better have a lower error rate, even with the results themselves being volatile and in a pretty small sample size. We only have 13 of the underachievers returning with enough playing time, thanks to pitchers with high home runs allowed frequently either being injured or quickly losing their jobs. ZiPS was closer on eight of the 13 (MAE 2.0% vs. 2.9%). For all pitchers, not just leaders and trailers, ZiPS had an MAE of 1.4% compared to 2.4% for actual home run rate.

zBB beat actual walk rate on 14 of the 20 returning overachievers and 14 of the 18 underachievers. The MAEs for the zStats were both lower as well (1.9% vs. 2.9% and 2.4% vs. 3.3%). The same pattern held true for all pitchers, at 2.1% vs. 3.1%.

For the overachievers, zSO was closer to results than the actual strikeouts were for 13 of the 19 players (3.7% vs. 5.4%). I was concerned about underachievers, given that a lot of the pitchers involved had injury questions, but ZiPS still eked out wins here both in roughly being closer (eight of 14) and MAE (4.4% vs. 4.9%). For all pitchers, the MAE was 3.9% vs. 5.1%. Since I’ve been a long time Kevin Gausman obsessee, I’m happy to have zSO miss on that one.

This is where we tie everything together, and it’s not an easy matchup considering one of the benefits of FIP is supposed to be its lack of volatility. zFIP was closer on 21 of 24 overachievers (MAE of 1.04 vs. 1.61) and on nine of 17 underachievers (MAE of 1.01 vs. 1.23). For all pitchers, the MAE was 1.03 for zFIP, 1.52 for FIP).

On a year-to-year basis, looking at players with 200 TBF in consecutive seasons (about 1500 players), ZiPS edges out the real stats in all four categories. The MAE edges are 0.9% vs. 1.1% in homer rate, 1.6% vs. 1.8% in strikeout rate, 3.3% vs. 3.5% in walk rate, and 0.72 against 0.82 in FIP. Naturally, zStats combined with the actual ones does even better than either category individually. Based on history, I modeled the ideal linear mix of zStats and actual numbers in predicting next year’s stats. For homers, it’s 92% zStats vs. 8% actual. While that might seem shocking, it’s simply due to the fact that home runs for pitchers are incredibly volatile, which is why xFIP improves on FIP just by assuming that homers are given up at a league-average rate by all pitchers, an intentionally preposterous scenario. For walks, it’s 72% zBB vs. actual. For strikeouts, the tally comes in at 57% zSO, 43% actual. As one final reminder, zStats contain no element of regression toward the mean; it only evaluates the Statcast/plate discipline data for the time periods in question.

Tomorrow, I’ll have the up-to-date zStat leaderboards for the pitchers.