Looking for Positional Bias in Prospect Rankings

Earlier this week, I focused on creating objective measures by which to examine and value individual prospects and farm systems. Inherent in those objective measures is the knowledge that the rankings themselves are not. Prospect writers like our own Eric Longenhagen and Kiley McDaniel combine in-person scouting with their knowledge and experience of the game, information from industry sources, and statistical data to arrive at well-informed but still subjective rankings and grades. What follows is one study attempting to determine if there has been any historical bias based on the position of a player.

As with the prior studies, I’m using the Baseball America Top 100 rankings from 1996 to 2010. To get a sense of how players were ranked by position, here are the raw numbers for the number of players listed at any given position, with multi-position players listed at both positions.

It should come as no surprise to find pitchers and outfielders first given that they have more starting positions available to them. Generally, a pitcher isn’t going to make a prospect list if the person believes he will be a reliever because the value is less. That might be changing some now, but for the vast majority of Top-100 pitching prospects, the hope is that they will be starters. If we were to divide the pitchers by the five starting rotation slots and outfielders by the three starting spots, shortstops would then have the highest representation on prospect lists. After shortstops, we have pitchers and third basemen, with outfielders slotting in ahead of first basemen and catchers with second basemen way down the list. Conventional wisdom holds that ranking so many shortstops is acceptable because many will eventually slide down the defensive spectrum, taking up slots at third base or over at second, which makes up for the lack of prospects there.

We can test the conventional wisdom pretty easily by looking at the performance of the prospects. The next graph shows the average present-day WAR totals for players by position.

Even with their strong representation on Top 100 lists, shortstops still perform very well. Third basemen come out on top by a decent margin, while second basemen–who boast fewer Top 100 prospects to begin with and seemingly perform worse than other positions if they do make it to the majors–come out last among position players, with pitchers coming in behind them.

We can take a look at how these figures come about by examining the number of busts and stars at each position, as the table below shows.

Shortstops end up at an All-Star level more often than any other position, confirming the conventional wisdom. Interestingly, first baseman don’t often end up as stars, but they are very often at least MLB contributors given their extremely low bust rate. Third base strikes a balance with a good star rate and low bust rate, while second basemen just haven’t done that much when ranked.

What the last table doesn’t factor in is the actual prospect rank. If second baseman and pitchers litter the bottom end of the rankings, then less is going to be expected of them. We can account for those rankings a few different ways. The first is to provide a little context by looking at the average ranking by position, as seen in the table below.

The reason second basemen produce less value is because they are generally ranked lower. First basemen come up the highest on the rankings, which might explain why their bust rate is so low. The bust rates come pretty close to mirroring the average prospect rank, with catchers the only position really out of place. While a lot of shortstops are ranked in Top 100s, they aren’t necessarily being pushed to the top of the list.

To go a little further, we can use an expected value by position based on their ranking/grade and compare to see which positions are doing better or worse with their actual production. To come up with expected production, I used the rankings and values from the updated prospect valuations, as seen below.

When we pair the prospect’s expected value with their actual value, we can see if there are any positions that have been under- or over-valued in the past.

I think the most important thing to note is that there aren’t many major discrepancies here. Six of the eight positions are within a couple tenths of present-day WAR value. Catcher is on the low end, though it isn’t too hard to chalk that up to WAR not completely capturing the value a catcher brings to the field defensively. I think coming into the project, I had concerns that positions like shortstop, or maybe left-handed pitchers, might be overrated by prospect rankings. While I can’t speak for all rankings, that certainly wasn’t the case for Baseball America from 1996-2010. It’s possible they slightly underrated shortstops and lefty pitchers. Outfielders and first baseman had very high expected outcomes and they came pretty close to living up to that billing, while second baseman were as expected.

The biggest outlier above is at third base, a position that has been pretty consistently underrated in baseball history. While shortstops have the ability to move down the defensive spectrum and still contribute in the majors, the same is true for third baseman as well. The third base rankings feature a lot of players who stuck at third base like Adrian Beltre, Eric Chavez, Evan Longoria, Scott Rolen, and David Wright, but there are also a lot of big stars who came up as third basemen–or were rated as prospects at third base–and eventually became important contributors elsewhere. Mark Teixeira, Albert Pujols, and Ryan Braun were third base prospects. Miguel Cabrera played third base before eventually moving across the diamond.

This is just one study, but it reveals that there doesn’t look to be much positional bias when it comes to ranking prospects. You won’t find many second basemen on prospect lists, but those who have been ranked haven’t performed well enough to think there should be more represented. Most of the positions comes in right where you’d expect them given their eventual value. It’s possible third base has been a little underrepresented, but that might be an anomaly based on a few Hall of Fame-level players. From 1996-2010, Baseball America seems to have gotten it right when it comes representing the positions in their rankings.