Confessions of a Baseball Analytics Writer

Jack Leiter will always have a special place in my heart. The Rangers’ top pitching prospect was the subject of the very first article I wrote for FanGraphs, which talked about, among other things, the unbelievable carry on his fastball and how it could lead him to big-league success. But we haven’t checked in on Leiter in a while, and well, his Double-A numbers have been ghastly: a 6.24 ERA in 53.1 innings pitched has somewhat muted the hype surrounding the righty. Though it doesn’t really change our outlook on Leiter, it’s still unsettling to see.

Part of that has been his inability to throw strikes, as Leiter is issuing well over five walks per nine innings. But more importantly, Leiter has lost a significant amount of his signature fastball ride in pro ball. Statcast data was available for this year’s Futures Game, during which Leiter’s dozen or so fastballs averaged 16.1 inches of vertical break – a far cry from the 19.9 inches I calculated in that debut article using TrackMan data. It could be a small sample quirk, and yet, the general industry consensus is that Leiter’s fastball is no longer transcendent. That’s a genuine problem.

What might the reason be? Maybe Vanderbilt’s TrackMan device wasn’t properly calibrated (as suggested by Mason McRae), leading to imprecise readings. But if that’s true (and maybe it isn’t), how could we verify it? What I came up with this: Using velocity, spin rate, and spin axis data from the 2021 NCAA Division-I baseball season, I built a model that estimates the vertical break of four-seam fastballs from righty pitchers. Once completed, I grouped the data by the pitcher’s team and looked at which schools over- or under-shot the model. Those with the largest residuals, in theory, are prime suspects for having miscalibrated TrackMan devices.

We have some evidence here. Among the schools with at least 2,000 righty fastballs in the database, Vanderbilt ranks ninth out of 48 in the average difference between actual and expected vertical break. As for Leiter himself? Across a not-so-small sample of 721 heaters, he generated 2.5 inches of extra ride over expected, which puts him squarely outside the confidence interval. It could also be that Leiter just isn’t throwing his fastball like he used to, but it does seem like TrackMan data had a hand in sweetening his statistical profile.

Even with diminished ride, Leiter’s fastball is still a plus pitch, and on the whole, he’s still one heck of a pitching prospect. But even in an era of sophisticated data, inaccuracies can be surprisingly common. TrackMan devices are operated and maintained by humans, after all, and to err is human. While having the requisite data remains incredibly helpful, a healthy dose of skepticism – and subsequent adjustments, such as removing outliers – goes a long way in making the most of it.

***

Making sure we aren’t being misled by the data is one thing. Deciding how to represent and communicate it is another. Lately, I’ve been writing a lot of articles about pitching, and a few of the comments expressed confusion over how pitch movement is indicated. As if baseball isn’t complicated enough, there is indeed more than one way to accomplish a seemingly simple task.

Because life is short and precious, here are the Cliff Notes. My preference is what’s known as “short-form” movement, or the expression of pitch movement relative to a pitch with zero spin-induced movement. Fastballs “rise” relative to that designated origin point, while breaking balls drop instead. Short-form movement reflects how hitters actually perceive pitches, as the illusion of rise is what beguiles them into swinging under high-spin heaters. It also creates a clear distinction between pitch types and how they behave, preventing us from mistaking changeups for sliders, for example. Short-form movement is what you’ll see on Baseball Prospectus (including Brooks Baseball) and this very site.

Then there’s “long-form” movement, which reflects how pitches move in real life. Fastballs still drop, but much less compared to breaking balls. This is what you’ll find over at Baseball Savant. I assume folks get confused because popular sites are using different methods of representing pitch movement, which is beyond understandable. But wait, there are even two types of short-form movement! The first, which comes courtesy of PITCHf/x, is measured 40 feet from home plate. The second, which comes courtesy of Statcast, is based on the entire flight path: 60.5 feet, minus the pitcher’s extension. They’re functionally the same, but one produces higher movement numbers than the other. More to the point, it makes our head hurt.

Life would be much easier if we could all agree on a single measurement, but given the sport we’ve chosen to arduously follow – how can you not be pedantic about baseball? – that’s probably not happening anytime soon. It’s not just pitch movement that’s drowning in semantics: Baseball’s trendiest breaking pitch is widely known as a “sweeper,” but in Yankee-land, it’s better known as a “whirly.” Spin efficiency (Rapsodo) is active spin (Baseball Savant), but some analysts take offense to the former, which implies the higher the efficiency, the better. Meanwhile, spin direction and spin axis are two entirely different things, but that’s scantly explained, so even smart writers will end up using them interchangeably.

Admittedly, I’m also part of the problem. On occasion, I’ll flip-flop between short- and long-term movement depending on what’s more convenient, in addition to omitting explanations that I assume just aren’t necessary. The truth is, there might be thousands of fans who aren’t as well-versed in baseball analytics as you think. It’s our responsibility, then, to make sure they’re accounted for.

***

As a FanGraphs contributor, there’s a certain amount of pressure to get things right, given the site’s reputation and amount of traffic. It doesn’t dawn on me like it used to, thankfully, but it’s still there in the back of my mind. Not that it’s a major issue – if you care about what you do, I think feeling at least a bit ashamed of a notable mistake is inevitable.

But you learn not to let those moments get a hold of you. You also learn that they present great opportunities to improve as a writer and an analyst. Earlier this month, I wrote about this season’s most and least consistent hitters, as determined by a series of calculations that I sufficiently explained and justified… or so I thought. Much to my dismay, someone in the comments pointed out that I had failed to normalize the hitters’ standard deviations in wRC+ based on their mean wRC+. Not doing so created a positive relationship between the two variables, from which many of the article’s conclusions were drawn. Ouch.

After review, I realized that, yes, I had made a fairly big mistake. There’s not much use in starting over with a new article, but I can make up for it here. First, below are the most consistent hitters, as of that writing, according to the normalized standard deviation in wRC+ (that’s regular standard deviation divided by mean wRC+, aka the coefficient of variation):

Next, here are the least consistent hitters:

There is some overlap: Alonso, Flores, and Wisdom are still in the top three in terms of consistency, and Miller remains mysteriously mercurial. Based on how many of the consistent hitters from last time have stuck around, much of where the normalization has played a role is in distinguishing actual streakiness from mere variance. Indeed, you’ll see that the most inconsistent list is no longer a list of the greatest hitters, which in retrospect didn’t make a whole lot of sense.

Still, adjusted standard deviation has a moderate correlation with overall wRC+, which suggests that good hitters really do tend to produce through streaks of brilliance. What Alonso and Co. are accomplishing remains special, albeit to a lesser extent. The correlation between standard deviation and strikeout rate is no longer nonexistent, but it’s weak enough to the point where it doesn’t warrant discussion. Case in point: Wisdom and Duvall, who occupy opposite ends of the consistency spectrum, are number one and three in strikeout rate respectively.

The takeaways aren’t dramatically different, but the names sure are. I’m disappointed for not having been more vigilant about how I presented the data before filing the article, but what’s done is done, and there’s this little follow-up to address what went wrong. While it would have been easier to ignore it altogether, I owe it to whoever is reading my work to be honest and self-reflective. After all, nobody wants to follow an analyst who pretends they’re right all the time.