Why Wasn’t Kole Calhoun a Prospect?

In the grand scheme of things, evaluating a baseball player’s current ability isn’t that difficult. If you have a sampling of their statistics and have a chance to watch a handful of their games, a reasonably informed observer will come up with a pretty good estimate of the player’s current talent level. Figuring out how good a player is right now requires some skill, but it’s nothing compared to having that same information about a player and attempting to forecast six years into the future.

That’s the job of a scout. You get a handful of looks at a 19 year old, you talk to a few people, you glance at some numbers, and then you’re asked to predict what that player is going to look like at 25. It’s a big ask and as a result, there’s a pretty big margin of error. A good scout should be more accurate than a decently competent observer, but no one would expect a scout to be right on every player. There’s just too much uncertainty involved.

One would hope that a scout doesn’t miss any Mike Trouts or bet high on many Jeff Larishs, but they’re going to miss good players and speak highly of bad one’s during the course of their careers. That’s especially true with respect to very young players.

The further a player is from the majors, the less certainty you should have in their future. A scouting report on an 18 year old with a half season of rookie ball comes with less certainty than one for a 23 year old on the cusp of the show with 2,000 minor league plate appearances to his name.

It’s one thing to fail to notice the potential in the 18 year old, but if he tears up every level over the next three seasons and you still haven’t caught on, there may be a flaw in your method. No one would fault you for missing him in high school, but if tremendous play in the minors doesn’t lead you to update your long term projection, you should probably recalibrate.

And the thing of it is, the vast majority of scouts and scouting writers do this very well. Some are obviously better at it than others, but the big name writers catch on, even when they don’t catch on right away. It’s part of their job and it’s what makes them good at it. You can’t survive only on your scouting chops, you have to make sure you’re constantly checking in on all of your evaluations to make sure they still hold water.

Yet there are still players who don’t get noticed and there’s one in particular that’s caught my eye. Why didn’t anyone notice Kole Calhoun?

Let’s start by painting two very clear pictures. Calhoun has had a great major league career to date and his minor league numbers were also quite good. He has 4.7 WAR in 784 career PA, built on a 122 wRC+ and not on any type of outrageous defensive numbers. That’s not a huge sample size, but he’s been consistently productive in the majors since coming up for good in the middle of 2013.

In the minors, he had a 156 wRC+ in rookie ball, a 143 wRC+ in High A, and a 147 wRC+ in Triple A. Calhoun didn’t just show up in the majors and start performing, he tore up the minors for three plus years and then showed up in the majors hitting 20% better than average. If you only knew about Calhoun’s statistics, you’d probably figure he spent some time in the top half of most Top 100 lists.

Instead, Calhoun was drafted 264th overall in 2010 and didn’t get an ounce of prospect attention anywhere but on Angels specific top ten lists, and only because the Angels’ farm system was such a wasteland during the early 2010s. No one had Calhoun on their radar.

Chris St. John creates a yearly consensus prospect ranking in which he averages out major rankings to offer the top 150 or 200 prospects in the game. Kole Calhoun was never listed. He never made the Baseball Prospectus list. He didn’t show up on FanGraphs’ list. Or Keith Law’s. Or Baseball America’s. When you look specifically at Angels’ list from that era, he’s usually about seventh. Calhoun wasn’t just underrated, he wasn’t rated at all.

No one thought Kole Calhoun was a prospect and no one thought he’d be an above average regular. The writers who do mention Calhoun at all talk about his lack of tools and ceiling and discuss him as a guy who might wind up as a serviceable fourth outfielder.

Marc Hulet, at FanGraphs before the 2013 season, had Calhoun 12th among Angels’ prospects:

A stint in the California League in 2011 did wonders for Calhoun’s prospect status but it may have also caused some people to over-estimate his ceiling. The left-handed hitter has the potential to be a very useful big league outfielder but he’s more of an above-average back-up than a starter on a first-division club.

Calhoun hustles his butt off and has hit everywhere that he’s played — outside of a brief 21-game sampling with the big league club in 2012. He does a little bit of everything but none of his tools stand out. He has a little pop, he can steal a few bases and he goes all out in the outfield while backing up all three positions. Solid depth in the outfield at the big league level means that Calhoun should open the year in triple-A but he could be the first outfielder recalled in the event of an injury.

Keith Law had him 7th on the team he believed had the worst farm system in baseball, stopping only to say Calhoun would make a great bench guy and extra outfielder. John Sickels also placed Calhoun 7th and said the same:

Former Arizona State star lacks big tools but has good instincts and polish, hit .298/.369/.507 in Triple-A. Decent on-base skills with occasional pop and underrated glovework, would be an ideal fourth outfielder.

A few Angels specific sites with smaller reaches were a bit rosier. The Angels Win Blog liked him as the team’s second best prospect by mid-2013, and Garrett Wilson of Monkey with a Halo ranked him fifth, pointing out that when he saw Calhoun play, scouts from other teams took notice. Wilson thought Calhoun’s game was complete enough to peg him for something more than a fourth outfielder, but that’s as high as anyone was on former Sun Devil.

Calhoun hit at every level and yet he didn’t even rise to the top of the Angels’ farm system, much less arriving on anyone’s Top 100. How can something like this happen? Why did so many smart people fail to recognize a player who hadn’t failed anywhere?

And if you’re rushing to the punch line, ZiPS wasn’t exactly on Calhoun either. Before 2013, the system pegged him for a .292 wOBA and 0.8 WAR. After his solid showing that year, ZiPS gave him a .319 wOBA and 2.1 WAR for 2014. That’s a nice bump, but it was still being very conservative.

After 2014, the projections are starting to come around. Steamer and ZiPS peg him for 118-122 wRC+ and about 3.0 WAR. The Fans are more bullish on his offense, as they tend to be, and like him for 132 wRC+ and 4.3 WAR.

Finally, the projections and the fans have decided that Calhoun is a good player but you also have to wonder, what took so long?

In the minors, Calhoun hit for power, he got on base, and he didn’t strike out a huge amount. He stole bases (he did get caught a lot in 2011, though). There’s some disagreement about the quality of his defense, but the debate ranges from a little below average to a little above. He got the chance to play in Salt Lake City and the PCL a decent amount, but he hit almost 50% better than the league while in AAA. There’s some penalty worth applying for the run environment, but not one that makes him into a non-prospect.

And if you watch the major league version of Calhoun, there’s nothing about his swing that troubles you, so we can probably infer that there wasn’t some obvious flaw during his minor league career. If you look at the facts of the case, there’s no good explanation for why Calhoun, specifically, didn’t get attention coming up.

Which means we’re dealing with a more systemic issue. I have three theories for how this happened, none of which are empirically verifiable by outsiders, but all of which speak to something about the nature of forecasting the future.

First, it’s always possible that this was just one of those random things that happens when you deal with lots of data points. There are thousands of players in professional baseball at one time and players cycle into the system every year. From a statistical perspective, everyone is bound to make random errors and sometimes those errors will overlap. If we reran history again, maybe Calhoun ranks 40th on a bunch of lists and we all forget this ever happened.

Second, I think it’s possible that there’s a clustering effect at play. A lot of prospect analysis, given the number of players to consider, is based on following up on the work of other people. If one or two trustworthy sources saw Calhoun look rough early in his professional career, it’s not crazy to think a certain opinion would start to take hold.

I’m not arguing that scouts were clustering to the opinion of a single scout and it clouded their own analysis, I’m saying it’s possible they got a negative report on him and simply never heard anyone else offer a good enough one to force them to go scout him themselves. Maybe a small contingent of scouts was coloring the waters in a disproportionate way.

But it’s the third theory that I think is the most interesting option. I don’t know if it’s the most likely, but what if the team scouts did this on purpose?

While the prospect writers file their own reports, a lot of what they do is informed by conversations with industry sources. To get a full picture on a guy like Calhoun, they would have talked to their sources within the Angels organization and with rival teams that scouted the Angels’ players.

Is it possible that these sources were overly negative on Calhoun with the intent to keep his ability under wraps? It sounds a little farfetched at first, but a rival team that likes Calhoun had no reason to spread around a glowing scouting report and turn the rest of the industry on to him, and the Angels scouts might have chosen not to trumpet his ability to keep him from winding up on the trading block due to the outfield logjam that existed there. This would have required some alignment of the stars, but nothing that rises to the level of conspiracy. When a player lacks eye-popping tools, there’s some room for deception.

On one hand, we have a player who hit his way through every level and succeeded at the major league level as soon as he was given a chance. On the other, we have a group of very smart people who were essentially blindsided by it. None of the industry leaders gave a thought to Calhoun as an above average regular and only one Angels blogger even hinted at a league average future.

Calhoun is good and has always been good, yet no one noticed. Those are two very different realities. Figuring out exactly what happened is impossible, but it could be as simple as a collective error or a flaw in the way scouts study players who lack loud tools. Or it could be as complicated as an effort to intentionally limit his notoriety. It’s a dumbfounding reality, which means the answer isn’t readily apparent. No matter the cause of the confusion, Calhoun should have been a prospect and aspiring scouts should be looking for more eighth rounders like him.