The Black Swan Theory of Drafting Pitchers

I wrote yesterday about the how the shelf life of draft rankings affects the finished product, using my “guy” from this year’s draft, Vanderbilt righty Carson Fulmer, as an example of a guy typically under-appreciated by this process. My history of scouting Fulmer goes back four years to his high school days, but my history of zeroing in on this type of pitcher goes back eight years.

Taking a Page from Wall Street

Nassim Taleb’s The Black Swan came out in 2007 and I read it toward the end of that year. Taleb made a lot of money during the stock market crash in 1987 and again during the financial crisis that started in 2007, a crisis he predicted in The Black Swan. The way he made his money is the underpinning of the book: better understanding how very rare events happen.

The human brain simplifies complex situations, which can often help us and conserve energy, but also makes us vulnerable when a seemingly unimportant piece of information is smoothed over by many individuals. Taleb names the unlikely event that few see coming a Black Swan, referring to the collective surprise exhibited when a black version of the (presumed exclusively) white bird was found in another part of the world.

In the book, Taleb lays out two buckets into which everything falls and to which he gives country-like names: Mediocristan and Extremistan. In Mediocristan, things fall on a bell curve, are generally pretty predictable and Black Swans aren’t really a factor. As an example, Taleb asks the reader to imagine a room of 100 random people into which the heaviest human on earth then walks. That’s outside of the norm, but the average weight of the room barely changes, as this one person’s weight only accounts for a small fraction of the weight of the room. Outliers have less power in this sort of environment.

Extremistan is the more interesting side of this coin. The example he gives there is the average wealth of this same room. Imagine how that number changes after Bill Gates walks in. It’s an unforeseeable, longest of longshots that Gates is the random person who walks in, but that fact changes the whole game: Gates accounts for almost all of the wealth in the room. Taleb also mention how book sales in a room full of authors might be affected if J.K. Rowling enters.

Taleb uses this framework to point out how the warm blanket of seemingly infallible financial formulas blind us to their hidden downside, when theories are taken to their logical extreme, all when working in the very Extremistan world of high finance. The takeaway, in Taleb’s words: “In Extremistan, inequalities are such that one single observation can disproportionately impact the aggregate, or the total.” The idea is that when you’re dealing in this world, being aware of the odds of a very unlikely event gives you a leg up on the masses who drastically underrate these long shot events to essentially be impossible.

Translating It to Baseball

This struck me as a principle that should be applicable to baseball, much like some of the principles discussed by Malcolm Gladwell in Blink, a book that was discussed in many progressive front offices when it came out. At the time, I was working in player development and scouting and thought this could best be used in the amateur player markets to refine the process of selecting players. The question was how to map amateur baseball players to fit in these two worlds and then figure out what the rubber-meets-road application would be.

This idea kicked around in my head for about five years, with versions of incomplete theories and hunches, but my initial idea is that the first round — or maybe even just the top half of the first round — is Extremistan and the rest of the draft is Mediocristan. The idea would be to look for players underrated by the traditional process to reap value with the top picks, then choose players that fit the rubric for big-league success in the lower rounds, when the stakes are lower and the talent more ordinary.

This idea for the later rounds, to steer into the middle of the bell curve and draft the players most similar to those that have already found success in the big leagues (height, weight, general toolset, etc.) is actually the approach taken by the Philadelphia Eagles with their whole draft board. Obviously football has some fundamental differences to baseball in its amateur talent market, but there’s some clear overlap in the general principles. Another excellent book about improving decision-making, Daniel Kahneman’s Thinking Fast and Slow, advocates for employing a simple formula in this sort of situation, which Grantland’s Chris Brown utillized to compare the top quarterbacks in this year’s draft:

Kahneman concludes that in situations where the odds of success are poor, holistic gut intuitions are most likely to fail, meaning decision-making can benefit most from a guiding formula. And since drafting quarterbacks fits the mold of the low-success enterprises Kahneman describes, it stands to reason that NFL teams would benefit from following a decision-making formula.

I haven’t created rubric for the lower rounds yet, since name recognition isn’t there for general fans and information can vary widely the lower you go down the draft board. That said, I will be doing it soon and there may be applications beyond the draft, but I’ll start there.

Defining the Cutoff

So, it made some sense how to tackle the middle-round player, but I was still a bit stumped on how to tackle the elite draft prospects and at what point in a given draft class the cutoff might occur and the method flip. I won’t reveal which team or the more specific outcomes of the study, but when I was with a club, a simple bit of research gave me an idea where the cutoff should be.

I took our overall pre-draft grades, a version of the FV grade that I use here, for multiple draft classes and plotted them against each other by grade and cumulative amount of players. The idea is to follow the lines from the bottom of the draft class up toward the top and see where the lines start to split apart. They basically overlap in the middle rounds, telling me the intuitive truth that, say in the fourth round each year, the pre-draft grades are the same across all drafts: the amount of players of fourth-round quality is essentially the same each year. The lines started separating around 15th to 20th overall, which is why I often state that the top half of the first round is the only thing that defines the actual depth of a draft. You can pick out a class of player (college infielders) and see more or less depth, but the overall quality of the whole class doesn’t differ.

This was also using just one club’s specific reports, even if they spanned multiple years. The results might be further smoothed and reveal that only the top-10 prospects define a given draft were industry consensus available, but I stuck with this top-15 to -20 range, as it’s specific enough and also makes intuitive sense that the elite players are in the top half of the first round, or maybe a little more in a deep draft. It’s also important to point out this is the top-15 or -20 players, not the top-15 to -20 picks. We know that, depending on the CBA and specific strategies that year, that a top 15-20 talent could go No. 1 overall, not sign, or go a couple rounds later.

Freestyling

So, now I’m trying to figure out what I’m looking for among the top-15 to -20 prospects in a given draft class that identifies them as Black Swans. One problem I found while trying to apply this concept to hitters is that they react to the action, which is dictated by pitchers. If we’re looking for underrated hitters, that would be difficult using this framework, because they are, by definition, chameleons who can hit all kinds of pitchers. If they’re rated among the elites in their draft class, they have some combination of very good bat speed and/or bat control and/or power. This ability either works at higher levels or doesn’t, but I’m not sure there’s clear stylistic differences that are systematically underrated by scouts. This feels like a Mediocristan-type situation where a scorecard of a half dozen abilities would be more useful.

Now, we’ve zeroed in on pitchers in the top half of the first round. As I mentioned above, there are all kinds and types of pitchers, varying by size, velocity, delivery, arm slot, repertoire and fly-ball/ground-ball style, among others variables. There could obviously be a type of pitcher who is systematically underrated by scouts, since there’s lots of ways to skin a cat when you’re dictating the action.  In addition, the success of pitchers is so heavily tied to injuries, that it would make sense were there a class of pitcher which proves itself to be more genetically fit in an evolutionary sense, and thus less likely as a class to get injured based on their type of on-field success.

If we’re looking for pitchers not appreciated by the typical scouting conventions that dictate the draft, we should start with small right-handed pitchers. If we want to have some confidence, particularly about future health, it would be good to have a track record of health and also a track record of success performing against good competition. Lastly, starting pitchers give more value than relievers and their more regular usage with lower stress mechanics gives us a better chance of capturing the most values. Plus, many of the best big-league relievers were starters in the minors, so that appears to be a higher class of talent.  So, we’ve zeroed in on small, right-handed, major-college starting pitchers with little to no injury history and a good performance record as the type of pitcher most likely to be Black Swans, using a combination of logic and the principles Taleb laid out.

The (Perfect) Examples

If you look at right-handed college starters under 6’0 drafted in the top-25 picks (as a proxy for top-15 to -20 talents, though I remember how guys were seen pre-draft in this window) in the last 10 drafts, you get four examples: Marcus Stroman, Mike Leake, Tim Lincecum and Sonny Gray.  That’s literally the whole list and that wasn’t cherry-picked to exclude a dud just outside those parameters. I also didn’t use only major college guys with no major arm injury history and good stats, but it turned out all four fit that description anyway. If you expand that to 6’0, you get Ian Kennedy and Trevor Bauer, but also Brad Lincoln and Hayden Simpson. Since Simpson was a well-under-slot pick that was seen as a one-round overdraft at the time, you could toss him out, since we’re limiting this to consensus top-15 to -20 talents. You could also argue that Phillies RHP Aaron Nola could be included in this group with Kennedy and Bauer.

The principle we can take from this is that players against whom the industry is biased (short righty starters) and who still are regarded as consensus top-20 prospects in their respective draft class (due in part to great performance at big-time college programs) turn out pretty well (six out of seven are big successes). It turns out that the more extreme the example (i.e. the shorter the pitcher), the more successful they are (four out of four). Using this evidence and the general principles above, you could start walking towards the idea that the amount of uniqueness (to use an awkward phrase) in a consensus first-round pitcher may be a good tie-breaker for whether he ought to be selected.

The problem for teams who are aware of this phenomenon and the reason it’s not a trend of which many teams are aware, is that this sort of player is rare, coming along only every year or two, as might one might expect from a thing referred to as a Black Swan. Also preventing a trend of Black Swans is baseball’s draft structure: major-league clubs can’t trade picks and to land one of these players requires a pick that’s high enough to have a chance to pick the player but not so high that a superior prospect is a better choice. Even if a team is aware of this, 100% sold on its predictive quality going forward and interested in leveraging this knowledge, they don’t really have any options to do that at this point.

Tying It Together

I kind of backed into it there, but this all fits with the general principles of what Taleb is pointing out and feels like the correct application of these methods to the baseball draft. The Black Swan search is obviously limited to a specific kind of player with specific kinds of picks, but we also get a framework for refining the process with picks past 20th overall, as well.

Another thing I like about this conclusion is the strain of natural selection. Justin Verlander and Tim Lincecum went in the top 10 of their draft classes and barely threw changeups as amateurs and had real command issues. Even they had to make a lot of adjustments in pro ball to become useful big-league pieces, and they were among the best in their respective draft classes. For Lincecum to get into that group given how truly weird he is at every level speaks to how fit he is in a evolutionary sense, as the scouting process is inclined to discount his abilities at every turn and to give Verlander every benefit of the doubt. If you draft room has run through every physical and general makeup and has two players deadlocked, things like ability to make adjustments are that last bit of information that can swing things one way or the other. You could then reason that height alone can tell us a lot about this ability, if both players are great college starting pitchers and similar in value as prospects.

If anyone you’re picking has to make significant adjustments and improvements after signing, would you rather bet on, all things being equal, the prototypical scout’s dream that’s been touted from day one or the underdog short dude who has no business being this high in the draft, except for his big stuff, command, clean injury history and great performances? The short guy has all kinds of biases keeping him out of this area of the draft, but he’s already proved himself fit for survival over and over by making adjustments and getting to this point. In other words, the scout’s dream just has to not be hurt and throw close to 90 mph and he’ll be fine, but the short guy has to break through all kinds of walls to get anyone to notice him. Which guy do you want to bet on making the necessary adjustments as quickly as possible, all else being equal?

Future Draft Classes

I can already guess the next questions: who are the Black Swan guys in this year’s draft, are there any guys that look like candidates in future drafts and how deep is the top group of talent in this year’s draft class. I’m still trying to figure out exactly how many guys belong in the top group this year, but my answer at this point ranges from 15-19 (most recent rankings). The very top group of talent is super shallow this year: I only gave a 55 FV to two players (full rankings coming soon), which would be in roughly the 30-75 range of a top-100 list. The sub-tier beyond that is of normal depth and everything beyond that is predictable as you’d assume from the above research.

In this year’s class, only Carson Fulmer fits the bill, but that makes this year a rare draft class because we have a clear example and this only comes every few years. I have Fulmer 6th in the class right now (he may move by a spot or two between now and draft day) and it looks like he’ll go somewhere from 5th to 9th this year (most recent mock draft), with some of the more enlightened clubs (the Astros, Red Sox and Cubs) showing the most interest in that range. As I wrote yesterday, some clubs don’t think he belongs in the top 15, but all the ones to whom I’ve spoken would at least stick him on the tail end of the elite group of talent.

As for the 2016 and 2017 drafts (rankings), there’s two candidates in 2016 in Florida RHP Logan Shore and Vanderbilt RHP Jordan Sheffield, but Shore is 6’1 and Sheffield is coming off of Tommy John with some real command issues, so neither is a perfect fit, but Shore makes the most sense at this point. For 2017, North Carolina RHP J.B. Bukauskas is the best candidate, listed at 6’0 but likely an inch or two below that. He’s like Sonny Gray in that he was a late first-round talent out of high school with big stuff: Gray tore his ACL before the draft so he opted to go to school to recoup his value and Bukauskas reclassified from the 2015 class to 2014 so he could go to school even sooner, with no intention of signing. Bukauskas is the best example of a potential Black Swan of these three, so if he can improve his performance and stay healthy in the next couple years, I’ll probably be pounding the table for him 24 months from now.

ADDED AFTER PUBLISHING: I realize that this doesn’t perfectly map to a true Black Swan situation like those presented in the book, like the creation of the Internet.  I used the principles from the book to find a meaningful conclusion that I’ve never heard anyone in the industry or media bring up before and the book was invaluable in helping me finding it. I find the evolutionary/injury prevention part of my justification more compelling from an academic standpoint, but both aspects brought me to this conclusion. I looked 20 players deep into 10 draft classes and there are 4 examples, which is 2% of the sample. That’s very rare, but not so rare that it should’ve gone unnoticed for this long.

If I wanted to find an example in the draft of something that very closely maps to examples in the book, we’d be talking about one player per 10 years at best, like Albert Pujols going in the 13th round and making the Hall of Fame, a level of shocking success that was essentially impossible to predict and hasn’t happened since 1999. At that point, an analysis would be useless, given the obscurity of the player pre-draft, the rare chance of being able to apply the knowledge and the tenure of the average scouting director (which is only a couple of years). So, yes, these pitchers are not Black Swans in the academic sense (some people took too much pleasure in pointing out this obvious point, such is life on the Internet), but that doesn’t invalidate the evidence, conclusions or process. Calling these players Black Swans is easier shorthand than using six qualifiers in front of the their name, so that’s what I’m calling them.