Dollar Sign on the Scout

I’m stating nothing new when I say that the popularity of Michael Lewis’s Moneyball did much to introduce its readers to the splendors of quantitative analysis in baseball. Nor is it inaccurate to say that Lewis — whose capacity for narrative is more or less unrivaled — characterized the sport’s older guard of talent evaluators (read: scouts) less as invaluable members of baseball’s front offices and more as mouth-breathing luddites.

For a number of reasons — most of them having to do with common sense — this image of scouts has disappeared almost entirely. Scouts are very clearly essential to the health of a baseball organization, and, generally speaking, it’s those teams that seek to use the best possible information — both visual and quantitative analysis — that experience the most success.

Still, even as the sabermetric community has acknowledged the importance of scouts and the act of scouting, there’s been no attempt (so far as I know) to measure the actual worth of individual scouts to their respective organizations.

This represents an attempt to do just that — to put a dollar sign on the scout, as it were.

Thanks to excellent work by Victor Wang, we have a sense of how productive a prospect might be (in terms of wins) given his place on Baseball America’s annual top-100 prospect list. Thanks to further work by colintj of Beyond the Boxscore, we know what the values of said prospects are in terms of WAR. These two sources represent the foundation of the present study.

To isolate the contributions of today’s scouts, I first started by taking BA’s top-100 prospect lists from each of the past five years (2006-10). For each of the prospects on those lists, I recorded (using a combination of Baseball America, the Baseball Cube, Cot’s Contracts, and reader input) both the bonuses paid out to said prospects and also the area scouts credited with their (i.e. the prospects’) respective signings.

Using Wang’s research (converted to WAR from WAB), I projected the likely yearly WAR totals for these prospects over their first six (i.e. cost-controlled) years. In cases where a prospect appeared on multiple lists (like, 2006 and 2007, for example), I took the highest of the extant rankings. Multiplying the WAR totals by $5MM (i.e. the current per-annum dollar amount per win), I was able to find a prospect’s “value” to his club.

After finding the values, I set out to find the costs of each prospect, too. While there are likely a number of costs associated with the development of a future major leaguer, the three main ones on which I settled were: (a) a player’s signing bonus, (b) three years of league-minimum salary (or, $1.2MM total), and (c) likely salaries during arbitration years (using the 40, 60, 80 method given a player’s projected yearly WAR values). Adding up these three components, I settled on the “cost” of each prospect to his respective organization.

To find the overall surplus value of a prospect to his organization, I subtracted the cost from the projected value. Using that method, here are the top-10 most valuable prospects from the last five years:

Roughly speaking, this is a list of top-10 batting prospects in inverse order of signing bonus, starting with Carlos Santana ($75K bonus), Desmond Jennings ($150K), and Mike Stanton ($475K). Nor is this shocking. For, while top-10 pitchers generally average around 6.00 WAR over the first six years of their MLB careers, top-10 batters average 13.50 WAR over the same span. This, of course, isn’t to say that hitters are always twice as valuable as pitchers, but rather that, owing to injury, etc., pitchers are much more subject to attrition.

To find the value of the scouts for this study, I connected the names and surplus values of all the prospects from the past five years to the scouts credited with their signings. Adding up the values connected to each scout, we get the following as a top-10 list — a sort of scouting all-star list:

So far as I know — and I’m more than willing to be corrected — here are those names with their respective organizations: Bill Buck, Detroit; Dave Jennings, Baltimore; Todd Blyleven, Colorado (although he left the organization to start a baseball academy in 2007 or ’08, I think); Sean O’Connor, San Francisco; Fred Costello, Arizona; Tim McDonnell, Florida; Jose Serra, Chicago NL; Ryan Fox, Florida then Washington; Fred Repke, Tampa Bay; and Randy Taylor, Texas.

Now that you see this list, there are certainly some caveats to make, as follows:

• Signing a player is not the business merely of an area scout. All the players you see on this list were also likely subject to the scrutiny of national crosscheckers, scouting directors, and general managers. In other words, signing a prospect is a team effort. In some cases, like with Justin Verlander, for example, who went second overall, the effort will be that much more communal. In any case, the point remains: for a scout’s name to appear on this list, he requires the support and trust of his organization.

• On that note, it’s also the case that sometimes multiple scouts are credited with the signing of a single player. How one deals with this is a question I’ve not answered perfectly. For example, four scouts (Ramon Pena, Ismael Cruz, Sandy Rosario, and Juan Mercado) were credited with the signing of Met farmhand Jennry Mejia. Do we attribute the surplus value of Mejia’s signing (ca. $13MM) to each scout in full? Do we give each scout a quarter of the value? For the present work, I’ve attributed the full signing to the first name on the list. So far as I can tell, none of the “supporting” scouts have been left off the top-10 list above as a result of this choice.

• By definition, we’re dealing with small samples here. Using five of BA’s top-100 lists means that there can be no more than 500 total prospects and no more than fifty top-10 hitters (i.e. the most valuable sort of prospect). In fact, there were about 340 total prospects, and the number of top-10 hitters is probably more like 25. Furthermore, no scout has more than four names appear on the combined lists, so each is being credited with value from just a few data points.

• This version of the study does not consider inflation — not in dollars per win, not in bonus values. So there’s no adjustment being made for a player like Verlander, for example, who made his debut in 2006, when a win was worth somewhere between $3.5M and $4.0M. Nor is there an adjustment made for players like Minnesota’s Aaron Hicks or San Diego’s Casey Kelly — i.e. players who’re unlikely to make their debuts in 2011. The idea has merely been to express all figures from the past five years in “present day” value.

• This study is only as good as the data that informs it. Baseball America (from whom I took most of the data regarding area scouts) is the gold standard so far as the business of prospect-mavening goes, but they’re subject to errors like anyone. Also, because I’m a careless, slovenly man, it’s possible that I, myself, have made manual errors.

• Please believe that I’m incredibly flexible so far as the methodology of this study goes. My main intent is to call attention to the work done by scouts and attempt to estimate what their worth might be to their respective organizations. I’m very clearly not the most talented of today’s baseballing analysts, but rather merely a curious fan/writer attempting to build on much more important work. As such, I regard this not as a definitive work, but as an attempt to start a conversation.

I’ll very likely treat the information here more fully in a later post. In the meantime, however, let’s acknowledge and recognize the names on this list.





Carson Cistulli has published a book of aphorisms called Spirited Ejaculations of a New Enthusiast.

45 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
eastsider
13 years ago

Very interesting. I think this is a great idea.

I am going to ask some dumb questions because I don’t really understand the scouting and signing process, but ignorance has never stopped me before.

How much credit should go to the scout for signing the top 100 prospects? It seems like it is pretty much agreed that these are the best, presumably even I could have seen that. If I get Tulo to sign, isn’t that more of a sign that my organization is willing to put up the signing bonus than my ability to evaluate talent? Would it be more interesting to find scouts who found diamonds in the rough in terms of late round draftees or foreign players?

rwperu34
13 years ago

You could subtract the expected value of their draft position/signing bonus.