How Many Good Players Were Good Prospects?
Usually you see this the other way around — how many good prospects became good players? It’s the foundation of any worthwhile prospect analysis, and based on the research, what we indeed observe is that higher-ranking prospects have worked out better than lower-ranking prospects, thereby granting validity to the prospect rankings themselves. If we didn’t see any differences in future performance, we’d have to think, welp, someone’s doing something wrong.
But when you focus just on the future of prospects, you ignore a massive part of the player pool — those players who weren’t considered good prospects. Now, professional baseball is selective for good baseball players. The majors are even more selective. Everyone with a job in baseball has a job because he has some amount of promise, and there’s no such thing as an untalented big-leaguer. But there are the guys who had a lot of hype, and there are the guys the hype never touched. So we return to the headline question: how many good players were good prospects?
There’s no perfect way to analyze this, but there is a simple way to analyze this. I decided to use a cutoff of 3 WAR. For position players, this was regular WAR; for pitchers, this was 50/50 FIP/RA9-WAR. So, for me, a good player was a player worth at least three wins above replacement. Now, about the prospect part. I used the Baseball America historical top-100 lists. They’re an authority within the industry, and they’ve made it easy by providing rankings going all the way back to 1990. So, for me, the definition of a good prospect was a top-100 prospect. I don’t care when the guy was a prospect; as long as he was a good prospect at least once, that counted.
I know that the top-100 doesn’t include all the good prospects. There are always intriguing players left off. But these are supposed to be the best 100 prospects, averaging more than three per organization, and when you get into an organization’s middle-tier prospects, they’re usually not expected to have big futures. They’re not supposed to become 3-WAR players. There are, to me, good prospects, more fringey prospects, and non-prospects. That’s an oversimplification, but the point is that, what I did is what I did, and now you can look at some numbers.
We’ll cover the last three years. The information for 2012:
In 2012, there were 114 players worth at least 3 WAR. So, by my definition, there were 114 “good” players. Of those, 54% had at one point ranked in the BA top 50. Another 18% had at one point ranked in the lower half of the BA top 100, so, 72% of the good players had previously been considered good prospects. The 3-to-1 ratio isn’t nuts — we know the better prospects have higher odds than the lower-ranking prospects. But then there’s the other 3-to-1 ratio — that of good prospects to unranked prospects. There were 32 3+ WAR players who never appeared on a list, and seven even had 5+ WAR seasons. I didn’t come into this expecting any particular number. A few years ago, 28% of good players hadn’t been too highly thought of as minor-leaguers.
Now, the information for 2013:
Now you should know enough to walk yourself through the data presented. There’s not very much of it. The unranked slice got bigger, at the expense of the 51 – 100 slice. This time, there were 134 3+ WAR players. That’s a jump of 20 from the year before.
At last, the information for 2014:
Check out that slice of the unranked! Out of 132 players worth at least 3 WAR, 51 from last season had never appeared on a BA top-100. So last year, 61% of good players had been good prospects. I don’t know if that’s more than you expected, or fewer, or exactly the same. It just is what it is. The 2014 numbers are different from the 2012 numbers, and it isn’t yet clear if that’s in any way meaningful.
Mike Trout? He was a prospect. Felix Hernandez? he was a prospect. Clayton Kershaw? Real good prospect. Yet there were 10 5+ WAR seasons posted by previously unranked players. The leader of all of them would be Corey Kluber. Kluber was at one point traded for Ryan Ludwick. He never showed up on a Padres top-10, nor did he show up on an Indians top-10. Then there’s Michael Brantley, a seventh-round pick who was at one point included in a trade as a PTBNL. He topped out at No. 5 on an Indians top-10. He never ranked with the Brewers. Jonathan Lucroy topped out at No. 5 on a Brewers top-10. You know that Josh Donaldson is an overachiever. Even Robinson Cano was never ranked; he was the No. 2 Yankees prospect before 2005, but that wasn’t considered a strong Yankees system. Cano was something of a tweener, and this future was never forecast for him.
Just averaging the last three years, you get this breakdown:
- 52% had ranked No. 1 – 50
- 15% had ranked No. 51 – 100
- 33% had been unranked
So to answer the question, based on the method explained above, two-thirds of good players were at one time good prospects. Which means one-third of good players were not considered good prospects. That doesn’t necessarily mean they were non-prospects — it just means they’ve overachieved, relative to prior expectations, and this might be of some consolation to fans whose teams don’t score real well in farm-system rankings. Good players are typically seen coming, to some degree. But, not all the time. Not even close to all the time. Players will surprise you, and there might be no better evidence than Kluber beating out Felix for the 2014 AL Cy Young. There is a whole lot of ways to be good, and there is a whole lot of ways to get better.
Jeff made Lookout Landing a thing, but he does not still write there about the Mariners. He does write here, sometimes about the Mariners, but usually not.
Awesome.
I wonder if the back end of top 100s tend to bet more on high risk/high upside teenagers in R/A ball while overlooking guys with higher floors who occasionally surprise, or maybe players who would have been comfortably in a top 100 prior to a year filled with injury or hiccuping on the first shot at a new level following a jump or skipped level.
Not just the back end. TOOLS! guys always populate a hefty percentage of any top 100 list due to their immense potential if they develop the skills to harness those talents. Javier Baez as the 5th best prospect and Archie Bradley as the 9th best last season are two good examples. Both already had glaring flaws and skill deficiencies, but their natural talent is so high that you have to acknowledge that potential. I imagine it’s the notion that skills can be taught whereas talent cannot that causes highly skilled players to be consistently underrated.
Same sentiment, but guys who are labeled as “pure athletes” often get higher rankings than they deserve. Baseball is the one sport where being the best athlete doesn’t really correlate to how good you are at the sport.
I don’t think it’s fair to say there’s no correlation. No direct relationship, sure. But being a good athlete is necessarily correlated with being a successful professional athlete.
I think these lists are also biased against players who make big leaps in skill sets during the course of a season, like Mookie Betts. If Betts had not been called up to the majors, he’d be sitting atop the prospect rankings right now; instead, his highest ranking is #75. That is, I don’t think unranked players lack ranking because they are “skilled” as opposed to “toolsy” — I don’t think it’s evaluator bias at work — but because players develop new skills after the rankings occur.
These lists are snapshots of player talent at a certain moment, and growth happens during the course of a year.
So would someone prefer to be unranked as opposed to 51-100?
No.
Probably just a numbers game. There are a hell of a lot more unranked players than players ranked 51 – 100.
Likely just a numbers game.
If the rate of decay from the top 50 to the second group of 50 remained true for the group representing all of the rest of ‘unranked prospects’ we’d expect a group of around 400 players to see such rates. This would represent team rankings from roughly 4-17 in an average system.
Since teams have more actual prospects than that, probably closer to 1,500 all together, I’d say the numbers imply that the likelihood of becoming a “good player” continues to decrease pretty significantly every step down.
Also I imagine a decent portion of the 51-100 of every list end up eventually being a 1-50 before they graduate.
That’s probably even more true of players that go on to be good.
Darn, I wanted a second look at some classy flat rimmed mesh backed baseball caps.
Interesting read. Perhaps would’ve been better if you’d crosschecked BA’s list with list from the likes of Keith Law and Baseball Prospectus. Now your article can also be a testament to BA’s relative incompetence, rather that the prospect status of a certain player.
Just not enough history, and certainly not organized as well as BA’s historical lists.
Baseball America is the preemptive source for prospects. No site or analyst goes into the depth that BA does. No disrespect to Kylie McDaniel, Keith Law, or BP. And as Jeff points out, they have more experience in the field.
I think that it shows just how competent BA is. Identifying 2/3 of all “good” MLB players while they are in the minor leagues is impressive. Especially given the huge numbers game they are up against by only selecting 100 each year (with lots of overlap).
By the way, I forgot to mention, this shows only a player’s highest ranking. Freddie Freeman, for example, ranked No. 17, No. 32, and No. 87, so I put him in the 1 – 50 group.
It’s a little bit funny, that’s actually what I thought you had said, I had to reread twice to see that you had not explicitly stated that. If I got it, I’m sure most prospecthounds got that too, instinctively.
And I enjoyed this analysis, it was pretty interesting. I’ve been curious about the failure rate in identifying good ballplayers since Pablo Sandoval was left off the Giants Top 30 list by BA the season he broke out and made the majors. While it was simple, I still think that it’s good analysis, good start to discussions along these lines.
I don’t think that it is fair for the commenter to call BA relatively incompetent, I think it’s a matter of it being really hard to identify good players, starting with the draft itself, then as players filter upward through the farm system. And for all we know, say, KLaw ends up missing more than BA if his rankings are analyzed.
And even this methodology has a flaw (not that you said it was perfect, but for discussion purposes). For example, Brandon Belt is probably considered a “win” using your methodology, but nobody saw how good he was going to be when he was drafted, other than perhaps the Giants, who selected him in the 5th round with post-analysis of their “overdrafting” of Belt. This is a rare public instance of their foresight, as a Giants scout was interviewed regarding their selections and the scout gave a glowing review on a player who expected to be drafted in the 10th round:
http://baseballbeginnings.com/2009/08/31/mapson-qa#more-2592
http://www.mercurynews.com/ci_17491142
Yet he is called a win once it was pretty obvious after his breakout season. And I realize that some obvious ones fail, Sean Bouroughs, Andy Marte, Andy LaRoche (traded for him in my keeper league …), but a lot of them succeed as well.
But we gotta start somewhere, and I’m glad you did the analysis and started the conversation about this. I don’t want to view it as a knock against baseball prospect analysts, but rather, getting a realistic view of how good they are at judging prospects. For it is not like baseball teams are all that good either.
And hopefully this will get widespread understanding that drafting and development is hella hard. Some people seem to think that it’s easy, but as my study 10 years ago on the draft first showed publicly, even in the first round, the success rate was very low in terms of finding good players once you got to the teams competitive for the playoffs, and even the worse teams weren’t having a coin flip’s chance of finding a good player. People get really angry about the draft, as if they know better, but even the people who actually have good overall information on all the amateurs don’t agree nor have a huge success rate. It’s all a crapshoot, a volume game.
I find that same sort of arrogance with prospecting in the minors as well. I expect studies like this to show that even the expert prospectors have issues identifying the good players, and hopefully most will cool off with the diatribes and angst and realize that everybody is feeling around the room blind, but some have a better finger on the pulse than others, and that the best you can do is compile what everyone is saying and make your best judgement, but be humble and modest, as reality will kick you onto your ass soon enough in the player prospecting business.
Any idea how many of those good players who were never ranked prospects were Japanese or Cuban players who skipped the minors and the prospect lists?
They don’t skip the prospect lists. Abreu was ranked. Darvish was ranked. Matsuzaka was ranked, and so on and so forth.
Generally, if they were any good, they would have been ranked in the off-season before their MLB debut, even if they skipped the minors and went straight to the majors. For example, most lists had Tanaka in their prospect lists last season, for a more recent example. And I’m sure Moncada will be added to lists once he signs with someone.
Interesting article. Of the +30% of previously unranked players I wonder how many have a disproportionate amount of their WAR derived from one particular skill or group of skills, such as defense & speed.
Personally without looking at the list, I bet they didn’t have one good to great tool or they would be noticed. They were just average across the board (David Dejesus comes to mind).
Which slice was Zobrist in?
I don’t think Zobrist was ever a top 100 guy, was he? So he’d fall into the unranked – unless I’m “mis-remembering.” I think Brignac was the expected to be the guy at SS for TB.
I was thinking the same thing. I totally see what Jeff says about no great tool, that makes a lot of sense.
The characteristic I was thinking, as that seems to be a factor in Giants prospects that people miss is the ability to make contact (i.e. don’t strike out that much). Sandoval and Panik were not highly thought of (relatively) despite that ability in the minors.
Generally, if a guy just has a good hit tool I think they don’t get ranked, unless they play one of the more challenging positions – C, SS, CF – and project to stick there in the majors. Or if they also project to hit for power as well.
Probably just noise, but its interesting that the percentage in the unranked group has grown the last two years.
Planning to follow up on this for JABO tomorrow, actually.
To detect a trend, it would be of interest to plot the average total mlb seasons played by 3 win players, as well as their minor league playing time.
What’s the distribution look like when broken out by pitchers vs. hitters. I’m speculating that there’s a much higher percent of unranked pitchers who develop into good players just due to the variability of pitcher performance.
I was hoping someone would make this comment. I agree completely.
It is the complete opposite. Good pitchers are easy to spot, not so with hitters.
This jives with what Kiley says as well, about the difficulty of projecting the hit tool
Jibes. JIBES!!!!
I speak jive!
haha, noted
This is very interesting stuff, and deserves a follow up article on the characteristics, if any, that the unranked good players group shares at higher rates than good players as a whole. I know you mentioned being rated average all-around, wonder what else.
That seems to be true, from my limited experiences watching the Giants promote their pitchers. For example, the Giants immediately after signing Bumgarner stated that they expected him to reach the majors in two years, and he did. They have also aggressively promoted some pitchers from the lower minors, like Jonathan Sanchez, Matt Cain, Tim Lincecum, though there have been those who faltered once reaching the majors after a quick ride up.
In addition, just from our base knowledge of how talent is distributed, things are different. Obviously, with hitters, we have no idea how they will handle certain pitchers, as the minors are populated with mostly with pitchers who don’t have the repertoire of MLB pitchers. Whereas, while that is true to an extent for pitchers as well, as the hitters are not as good, there are a lot of hitters who can handle fastballs in the minors and fastballs remain the key part of most pitchers’ arsenal, and thus the view of how they might make the leap to the majors involve less projection for pitchers, since one can see how opposing batters handle their best pitches, which tends to be fastballs.
I’m fairly sure that I wasn’t ranked this year, and I was starting to get worried that I was never gonna contribute 3+ WAR at the major league level. Good to see I still have a chance.
So you’re saying there’s a chance!
I would be interested to know how long did it take for a prospect (on average) to achieve an above average WAR? Would the results differ between the 1-50 and the 51-100 class?
I would’ve interested in having this broken down, per team! Even could go in depth within each team’s ranks, how many were out of the top X in the organizational rankings, etc.
Another factor might be the WAR cutoff. For example, Bruce Rondon grabbed 95 in 2012. How many relievers ever approach 3 WAR in their careers… Chapman, Kimbrel, and…
“Another factor might be the WAR cutoff. For example, Bruce Rondon grabbed 95 in 2012.”
Rondon’s 95 WAR season fits the 3 WAR cutoff so don’t worry about that. Also, I had no idea he reached 95 WAR in 2012. How did David Price win that year’s AL Cy Young?
Writers strongly suspected he used some sort of performance enhancer that year. My guess is gama rays or a radioactive spider bite.
Not being a top 100 prospect doesn’t mean you’re not a “good prospect”. Josh Donaldson is really the lone exception as to completes coming off the grid
Is there a way to look at average age in the groups when the player broke into the majors?
Speculation – Are teams pushing players (especially pitchers?) to the majors faster these days, making it more likely the player has not climbed into the top 100 or top 50 lists prior to cracking the bigs?
Also, is there a pitcher vs hitter shift in any of the data from year to year (specifically the unranked group)
I see where you’re coming from in terms of very quick risers, but I think this would actually work the opposite way. 100 players have to appear on the list each year. The faster teams push players to the big leagues, the fewer lists each individual player is to going to appear on and therefore that would increase the percentage of active players that would’ve appeared on the list at least once.
I’m not sure if I agree, or at least I don’t think it’s totally clear that you’re right. Cano is an exception to your theory — he was never ranked through 2005, jumped in and played 130 games with a 105 wRC+, and was never a prospect again.
Another really good player off the top of my head that came from nowhere was Goldschmidt. I don’t ever recall seeing him on a topp list anywhere (BA, BP, KL, Sickles, etc.).
Do we tend to favor age/level too much over production? I can understand it a little more at lower levels, but A+ and AA I think we have an idea on the tools and should be more curious about how the tools are developing and producing, not whether they tend to still have them.
Keith Law had Brantley in his top 100. This method seems a bit flawed considering the use of only one list. Prospects 75-150 are often considered interchangeable, so I would be more interested in seeing a study that uses all of BA, BP, Law, and Sickels rankings. I can’t say for sure, but I imagine the number of non ranked prospects would drop significantly if we used a wider range of prospect opinions.
Feel free to do your own analysis with as many lists as you’d like..
Well obviously it would if we include more than 100 guys each year. It comes down to numbers, I think these data are a good reflection on prospect lists, not a bad one.
Of course there would be a wider range, that’s just a function of incorporating more rankings.
I think it would be more interesting to use Jeff’s methodology and apply it to each of the sources you listed, BP, Law, Sickels, and see how they do vs. what BA did.
Also, defense is only now being valued appropriately on most prospect lists. Correct me if I am wrong, but a large part of WAR is related to a players defensive capabilities. Plenty of players boost their WAR on defense alone.
If only BA knew about this “defense” component you speak of…
A very good chunk of these nonprospects were likely in the top 101-250 prospects in the game. Had BA ranked more than 100, we’d know that for certain.
Yeah, an unscientific way to look at it would be to take each teams top tens. Not all systems are equal. But that would at least give you a pool of 300 players that were positively regarded, if not necessarily the top 300. I’d want to know how many 3+ win players didn’t make even a top ten list. I’d guess not a lot.
A better idea, would be to take all of the players who showed up on anyone of BA’s top 150 rankings (the top 100 is a consensus composed of 7 different writers). Each of the past 2 seasons (possibly more) BA has released all of the players names who were named on a top 150 list but failed to make the consensus top 100 list.
I’d be interested to know what the injury rate was for the ranked prospects..
Original and fantastic premise for a column! Thank you!
I think it pretty common perception that BA has done a very poor job of covering prospect lists recently. You can see it on the increasingly unranked 3 WAR players. That being said, it looks like they made real effort this year.
I would suggest that perception is dead wrong. They do a better job than anyone else IMHO.
I think it’s pretty common perception that [insert unsubstantiated opinion of some entity]. You can see it on the [insert data point with no context]. That being said, [insert back-handed compliment to aforementioned and disparaged entity].
In 2012, there were 114 players worth at least 3 WAR. So, by my definition, there were 114 “good” players. Of those, 54% had at one point ranked in the BA top 50.
How many players ranked in the BA top 50 WEREN’T worth 3 WAR?
If we don’t know that, this really means nothing.
Is the 114 all the players ever ranked in the top 50? Half? 10%?
Maybe you skipped over the very first sentence? The one where Jeff specifically says that “Usually you see this the other way around” and then explains why he thinks this particular analysis might be interesting?
He was wrong.
Hmmmmmmmmm. This feels forced. A few thoughts.
1. I think every year a huge percentage of top 100 prospects are right handed pitchers. At least 30 each year and as high as 34-39 I have seen. Another 10-15 percent are left handed pitchers. Pitching attrition due to injury/non-performance is pretty high. Relievers are often former top prospects and rarely rate highly by WAR.
2. Distribution of prospects is not uniform and therefore opportunity to play will vary base on the organizational strength, depth, and philosphy. Some organizations will have 7 and others will have zero. The dudes with 0-3 still promote prospects and let them play.
3. The argument that Baseball America’s list is more accessible is poor given you are not going back to 1992. It would have not taken long to aggregate multiple lists (as they vary greatly) given that the underlining premise is to question the validity of prospect lists. I would argue that Baseball America has lost significant credibility of late.
4. The findings actually support the value of the lists i.e. the higher the ranking the great the probability. Given that you must play to become valuable and at the highest level the difference between being good and bad is often tiny and playing time can change trajectories.
Anyway, interesting read. Thanks
The premise is definitely not to question the validity of prospect lists.
This is a topic I find interesting because many seem to dismiss prospect rankings because so many top prospects don’t pan out, which seems like a backwards way of assessing prospects lists to me.
Last year at Let’s Go Tribe I went through a somewhat similar process, concluding that while most top prospects DON’T turn into good players, most good players WERE top prospects. If anyone is interested in more reading on the topic, here’s what I wrote:
http://www.letsgotribe.com/2014/2/5/5379248/mlb-news-baseball-america-prospect-rankings?_ga=1.70241616.1766080274.1373922015
100% agreed. I have no idea if there is a widespread dismissal of rankings, but if there is i KNOW that I have never heard a reasonable alternative. I mean, the draft is a ranking. The tiered minors are rankings. “Your” favorite team is ranking them. And even “you” are subconciously ranking them…. So why piss on BA, et al for trying to systematize and set it in history?
This is a very useful analysis from a fantasy perspective. It suggests that you’re sliding down a very steep probability curve when you draft prospects down the rankings or off the rankings.
As to extensions of the analysis, I’d like to learn more about the shape of that curve. The top 50 cutoff is pretty arbitrary. Are the 3+ seasons concentrated among the top 10s or 20s of BA’s ranking, with a pretty flat probability thereafter? That’s what I would guess– it may be easy to separate a #1 from a #60, but maybe not so easy to distinguish a #30 from a #90. I look forward to reading of more along these lines…..
What percent of all players in each of those years are 1-50 prospects, 50-100 prospects, and unheralded as prospects? This is some nice initial work, but I think needs some expansion to really tell a story.