How to Identify 1.14 Excellent Pitchers at the Fall League

There are a lot of reasons to assume that right-handed Boston Red Sox prospect Michael Kopech is well positioned — or, at least, as well positioned as any young pitcher can be — not only to reach the majors but also to experience some success while there. The velocity is one reason why, obviously. He reportedly hit 105 mph on multiple radar guns earlier this summer. He was sitting at 93-97 during instructional-league play last month. He’s recorded an average fastball of 98.7 mph in the Arizona Fall League. Provided his arm remains attached and in mostly serviceable condition, he appears certain to serve at least as competent reliever.

There are other promising indicators, though. Objective ones. Consider: Kopech was selected 33rd overall in the 2014 draft. Research by Matthew Murphy indicates that players taken between 31st and 35th overall as amateurs produce about two and three wins over the course of their team-controlled years. As for his status as a prospect, specifically, one finds that Kopech was ranked 89th on last year’s edition of Baseball America’s top-100 list. The most recently updated iteration of Kevin Creagh and Steve DiMiceli’s work on prospect valuation suggests a pitcher ranked between 75 and 100 on BA’s list is likely to contribute something like three or four wins before reaching free agency.

Two or three or four wins might not seem like a great result over half a decade, but that’s also just an average figure. There’s a lot of attrition baked into it, a lot of zero-win seasons. The point is that, on average, a pitcher with Kopech’s profile ends up contributing in the majors. That has considerable value.

There’s one more promising indicator for Kopech, though, and it’s not necessarily one you’d expect — namely, his performance so far in the Arizona Fall League, where he’s recorded the highest strikeout rate among starters. Given the circumstances under which it’s played, there are multiple reasons to discount the numbers coming out of the AFL. It’s a developmental league, first of all. The mandate isn’t really for prospects to “win” as much as it is to improve. Plus there’s also the fact of the inflated run environment, which is probably the product of an imbalance in the quality of pitching and hitting prospects and also the product of the weather in Phoenix. Finally, there’s an even more basic reason why to approach Fall League data with some caution: the samples are small. There are only about 30 games in an AFL season — or, the equivalent of a month’s worth of regular-season major-league play. The dangers of extrapolating too much from a single month of data are manifest.

All of that is valid. That said, one shouldn’t immediately throw out all the statistical bath water quite yet, lest some figurative babies remain inside it. Because even after accounting for all the reasons why AFL data oughtn’t predict future success, there’s probably still some use for it.

A couple years ago, I examined a collection of AFL pitchers from the latter half of the aughts and found a real correlation between their strikeout rates in the Fall League and their eventual success (or lack of it) in the majors. Perhaps this isn’t shocking: strikeout rate is pretty elegant as an evaluative tool. Not only does it become reliable in much smaller samples than other metrics, but it exerts a great deal of influence on run prevention, also. If a pitcher is striking out opposing batters in the Fall League, that would seem to be a strong indication that he’d strike out other kinds of batters, too — which, because of how strikeouts work, would also lead to run prevention.

In any case, with a couple more years’ worth of data available now, it appears as though the correlation between AFL strikeouts rates and major-league run prevention remains. I’ll get to the numbers in a moment. First, here’s a lightly edited excerpt from that piece two years ago, detailing some of the methodology of this exercise.

To better understand how much pitcher strikeout rate in the AFL rate might inform future major-league success, I looked at AFL pitchers from 2005 to -09 who’d both (a) faced 70 or more batters (that is, the sample threshold at which strikeout rate becomes reliable for major-league pitchers*) and also (b) made at least half their appearances as starts. Each year in the AFL there are about 25-30 pitchers who meet both those criteria. For each of the five years in question, I isolated both the top and bottom third by AFL strikeout rate, resulting in 43 “high strikeout” and 43 “low strikeout” pitchers.

*As Russell Carleton notes, these thresholds oughtn’t be regarded as a “magic number,” but rather “the point where a measure of reliability slowly crosses an only-somewhat-arbitrary line in the sand.”

Yesterday, in my cold house, I expanded the original sample, including AFL pitcher-seasons from the 2010 and -11 campaigns, as well. Doing so adds 17 more pitcher seasons to both the “high strikeout” and “low strikeout” pitcher groups, bringing each sample to a total of 60 unique player-seasons. The differences between the major-league numbers of the two groups are substantial.

First, the high-strikeout group (sorted by WAR as calculated with runs allowed):

Next, the low-strikeout one:

There are a lot of names and numbers in these tables. Readers, as members of a free and just society, are welcome to interpret the data as they want. Here are some telling observations, however:

• Among the high-strikeout group, 45 of 60 pitchers (75%) have recorded at least one major-league appearance. Among the low-strikeout group, only 30 of 60 (or, 50%) have.
• Among the high-strikeout group, 7 of 60 pitchers (12%) have both (a) recorded the majority of their major-league appearances in a starting capacity and also (b) produced a strikeout rate above 20% (i.e. roughly the league-average rate). Only 2 of 60 (or, 3%) from the low-strikeout group (Mike Minor and Tyson Ross) have done the same.
• Pitchers from the high-strikeout group have compiled a collective 222.8 RA9-WAR, or 3.7 wins per pitcher. The low-strikeout group has produced just a 10.5 RA9-WAR collectively, or 0.2 wins per pitcher.
• Among the high-strikeout group, 8 of 60 pitchers (13%) have recorded a 10 RA9-WAR or better. Zero pitchers among the low-strikeout group have reached the 10-win threshold. (Indeed, with the exception of Minor and Ross, no low-strikeout pitcher has recorded more than two wins by this measure.)

Regarding that fourth and final observation: 10 wins is totally arbitrary as a point of demarcation. That is, there’s nothing appreciably different between a pitcher who’s produced 10.1 RA9-WAR over the course of his career and another who’s produced 9.9. That said, there are fewer than 100 active pitchers who’ve recorded more than 10 career wins. Reaching that 10-win threshold requires some combination either of peak or longevity for a pitcher. It’s either the result of five average seasons or two MVP-level ones or 20 half-win seasons. Whatever the case, it’s difficult for a pitcher to find his way there by accident. These pitchers are excellent*.

*More on the (unexpected but probably deserved) characterization of Josh Collmenter as excellent in a later post.

The seven-year sample of high-strikeout AFL pitchers considered here has produced eight of these major leaguers, or 1.14 per season (the figure cited in the title of this post). Perhaps that number sounds low. After all, the Fall League has a reputation for showcasing some of baseball’s best talent. And, in fact, the number is a bit low. Remember, we’ve only considered the high- and low-strikeout groups here. Adding in the third and final group — the pitchers who produced roughly average strikeout rates during their Fall League tenures — augments the collection of 10-win pitchers by five: Gavin Floyd (from the 2006 AFL season), Ricky Nolasco (2007), Matt Harrison (2007), Clay Buchholz (2008), and Mike Leake (2009). So that’s 13 different 10-win pitchers from a sample of 182 (or, roughly 7%) over seven years (or, about 1.63 per season).

The precise number of overall excellent major leaguers produced by the Fall League isn’t ultimately the point of this post, however. Rather, the focus here is on the predictive nature of strikeout rate in the AFL — and the addition of the average-strikeout group to the overall sample lends some credibility to it. As the data reveal, zero of the AFL’s low-strikeout pitchers have reached the 10-win threshold as a major leaguers and eight of the high-strikeout group have. If strikeout rate were actually predictive of future major-league success, one would expect the AFL’s average-strikeout group to have produced a figure somewhere between zero and eight. In fact, it has: it’s produced five of them, or almost exactly the halfway point. As for the relevance of this to the current AFL season, that’s a consideration for a different post. For the moment, however, the data seem to provide a framework by which to better identify future excellent major-league pitchers.