Crowdsourced vs. Actual Contract Values from 2013-2014

Later this morning, FanGraphs will publish the results of this year’s contract crowdsourcing effort, a list of the top-55 free agents (more or less) in order of total contract value as estimated by the wisdom of the crowd that is FanGraphs’ readership.

The projected contract values in that post will likely not precisely match the actual contracts received by this offseason’s free agents. To understand what biases the crowd might exhibit (if any), I’ve published below the results from last year’s crowdsourcing effort, alongside the actual contracts received by the relevant free agents.

YRS denote the length of the contract in years; AAV, the contract’s average annual value in dollars; and TOT, the overall value of the contract in question. Column headings preceded by a -c- represent crowdsourced values, while those preceded by an -a- represent actual values. Finally, in the rightmost columns, the letter -d- in the column headers represents the difference between the actual and crowdsourced value, where a positive figure indicates that the player received more than the crowd expected; a negative value, less than the crowd expected. So, for example, the crowd estimated that Robinson Cano would receive $186.4 million, while he actually received $240.0 million — i.e. $53.6 million, or 28.8%, more.

With regard to the acronyms at the bottom of the table, AVG represents the average of all figures in the relevant column; RMSE, the root-mean-square error. Root-mean-square error represents, in short, the average amount (either above or below) by which the crowd missed.

Players appear in order of actual total contract value, although the table itself is sortable. Contract information courtesy of MLB Trade Rumors.

A brief inspection of the table reveals two main observations:

• As has been the case in previous years, the crowd was generally conservative with regard to its estimates of the largest contracts. The offseason’s three most high-profile free agents — Robinson Cano, Jacoby Ellsbury, and Shin-Soo Choo — received ca. 40% more overall than the crowd expected (although the largest part of that figure is due to Choo’s contract, which was regarded as rather large at the time).

• Players received, on average, 11% fewer overall dollars than the crowd expected. Note, however, that merely adding 11% to every contract’s value doesn’t necessarily create a more accurate overall picture. In fact, subtracting 11% from every crowdsourced estimate actually leads to a larger RMSE (i.e. a larger average difference) between the crowdsourced and actual values. The reason: as noted above, the crowd’s misses weren’t evenly dispersed, but generally included underestimating the contracts given to more desirable free agents and, in many cases, overestimating the amount received by more ordinary free agents.

These points will likely be relevant to our newest data set, as well, when it’s released later this morning.