SCOUT: Using Small Samples from the AFL

For those sabermetrically oriented baseballing enthusiasts who enjoy moonlighting as prospect mavens, the Arizona Fall League — and the various Caribbean leagues, too, for that matter — represent a conundrum. On the one hand, it’s exciting to see so many young, talented players competing against one another; on the other, the small sample sizes prevent us from making authoritative statements about the respective players’ performances in said leagues, even though we’d like so badly to do just that.

Because of these limitations, we’re almost entirely at the mercy of the Bryans Smiths of the world. No, it’s not “mangy scoundrels” I’m talking about — although that’s certainly an appropriate description of him — but rather “authentic prospect mavens.” Certainly, Smith’s observations on the AFL are helpful, and first-hand accounts are preferable to numbers at this stage.

But if we really insist on ever using winter-league stats — even in the most offhanded fashion — allow me to propose a method by which we might use a hybrid of scouting and, uh, stats-ing.

By way of Russell Carleton’s (a.k.a. Pizza Cutter’s) often referenced, now archived post on the reliability of sample sizes in baseball, we learn that two of the three triple-slash stats — on-base and slugging percentages — don’t become reliable until around 500 plate appearances. That is, 500 plate appearances is the level at which, in Carleton’s words, the “stat can be considered to be saying something about an individual player.” As for batting average, Carleton found that it doesn’t become reliable until somewhere around 1000 PA.

Herein lies at least one of the problems with winter-league stats. Because the AFL leaders in plate appearances rarely top even the 125-PA threshold, we’re forced to regress them over two-thirds of the way back to league average. That creates little in the way of meaningful separation.

An alternative, however, is to look at those categories that (a) become reliable more quickly, but also (b) tell us the sorts of things we like to know about a prospect — namely, the quality of his tools. In this case, we can probably say at least something about contact- and power-hitting — via strikeout and home-run rate, respectively.

Per Carleton’s study, strikeout rate becomes reliable at the 150-PA threshold; home-run rate, at 300 PAs. Those numbers are more friendly than the necessary samples for the triple-slash stats. Even after regressing the remainder of the way* to league average, the resulting adjusted strikeout and home-run rates are distributed widely enough to help us make some kind of observation about player performance in Arizona.

*Which would be, using strikeout rate, for example, [[150 – PA * (Lg K-Rate)] + [PA * K-Rate]] / 150.

Note, please, that there are a number of caveats to make here. First, is that the player pool — i.e. almost exclusively young and developing players — might very well alter the reliability thresholds of the different metrics. Second, there’s the chance that, because the players are being observed by coaches and working in different roles than they might usually, that they are performing differently than they would under different circumstances. (See Chris Carpenter’s comments, noted in Bryan Smith’s recent AFL Notebook.) Third, the parks in Arizona are weird. Does that change things? I don’t know, entirely. But finally — on the subject of my fallibility — there’s the distinct possibility that I, Carson Cistulli, am the sort of person who knows just enough to be dangerous. I think I’m being responsible here, but I also think having garlic as a pizza topping counts as a serving of vegetables. So, that’s what you’re dealing with here.

With all that as preface, allow me to introduce what I’ll call SCOUT. To devise it, what I’ve done is to find the regressed strikeout and home-run rates (xK% and xHR%) for all the qualified batters in the AFL. Then, for each player, I’ve found the z-score (that is, standard deviations from the mean) in xK% and xHR%, and averaged them (i.e. the z-scores) together. SCOUT is the result of that.

By that method, here are are the current leaders in the AFL:

And the laggards, too:

What I think SCOUT is able to capture — and this is why I think it might have some value — is a couple of underlying skills that inform batting production. Like, consider the case of Conor Gillaspie. His slash-line (.231/.279/.410) is pretty bad. Still, given his strikeout rate (just 7.1% so far, unadjusted) and his pair of homers, we know that Gillaspie probably hasn’t had a horrible fall so far — at least so far as his contact and power tools go. SCOUT helps us see that, I think.

Conversely, among the laggards, we see that Kris Negron is having some success, posting a .282/.364/.538 thus far. But most of that is coming from batted-ball success that he’s unlikely to sustain. In fact, Negron has struck out in a just over a third of his plate appearances. Though one might be tempted to say that he’s off to a good start, there appears to be little reason to make that claim — besides his rather unreliable slash stats, that is. Perhaps, at the very least, we can say that Gillaspie is demonstrating greater success so far as his contact- and power-hitting tools go. That’s not a totally unhelpful remark.

What are SCOUT’s (limited) uses, ultimately? They’re two-fold, I think. First, I’ll be using the number — in addition to, maybe instead of, slash stats — in the offseason notes I’ll be providing here. Second, it can serve, I think, as the tiniest contribution to the much larger discussion over how we might use numbers to complement scouting.

Tomorrow, I’ll begin the offseason notes in earnest and begin looking at some of the leaders more closely.