Crowdsourced vs. Actual Contract Values from 2013-2014

Later this morning, FanGraphs will publish the results of this year’s contract crowdsourcing effort, a list of the top-55 free agents (more or less) in order of total contract value as estimated by the wisdom of the crowd that is FanGraphs’ readership.

The projected contract values in that post will likely not precisely match the actual contracts received by this offseason’s free agents. To understand what biases the crowd might exhibit (if any), I’ve published below the results from last year’s crowdsourcing effort, alongside the actual contracts received by the relevant free agents.

YRS denote the length of the contract in years; AAV, the contract’s average annual value in dollars; and TOT, the overall value of the contract in question. Column headings preceded by a -c- represent crowdsourced values, while those preceded by an -a- represent actual values. Finally, in the rightmost columns, the letter -d- in the column headers represents the difference between the actual and crowdsourced value, where a positive figure indicates that the player received more than the crowd expected; a negative value, less than the crowd expected. So, for example, the crowd estimated that Robinson Cano would receive $186.4 million, while he actually received $240.0 million — i.e. $53.6 million, or 28.8%, more.

With regard to the acronyms at the bottom of the table, AVG represents the average of all figures in the relevant column; RMSE, the root-mean-square error. Root-mean-square error represents, in short, the average amount (either above or below) by which the crowd missed.

Players appear in order of actual total contract value, although the table itself is sortable. Contract information courtesy of MLB Trade Rumors.

Player Pos cYRS cAAV cTOT ——– aYRS aAAV aTOT ——– dYRS dAAV dTOT %dTOT
Robinson Cano 2B 7.6 $24.5 $186.4 | 10 $24.0 $240.0 | 2.4 -$0.5 $53.6 28.8%
Jacoby Ellsbury CF 6.0 $18.7 $112.0 | 7 $21.9 $153.0 | 1.0 $3.1 $41.0 36.6%
Shin-Soo Choo COF 5.0 $16.2 $81.1 | 7 $18.6 $130.0 | 2.0 $2.4 $48.9 60.4%
Brian McCann C 4.4 $14.8 $65.4 | 5 $17.0 $85.0 | 0.6 $2.2 $19.6 30.1%
Curtis Granderson CF 3.5 $14.0 $49.0 | 4 $15.0 $60.0 | 0.5 $1.0 $11.0 22.4%
Ubaldo Jimenez SP 3.6 $12.2 $43.8 | 4 $12.5 $50.0 | 0.4 $0.3 $6.2 14.1%
Matt Garza SP 4.4 $14.7 $64.5 | 4 $12.5 $50.0 | -0.4 -$2.2 -$14.5 -22.4%
Ricky Nolasco SP 3.7 $12.6 $46.1 | 4 $12.3 $49.0 | 0.3 -$0.3 $2.9 6.2%
Carlos Beltran COF 2.2 $13.4 $30.1 | 3 $15.0 $45.0 | 0.8 $1.6 $14.9 49.4%
Tim Lincecum SP 3.0 $13.3 $40.5 | 2 $17.5 $35.0 | -1.0 $4.2 -$5.5 -13.5%
Mike Napoli 1B 2.8 $13.3 $37.3 | 2 $16.0 $32.0 | -0.8 $2.7 -$5.3 -14.1%
Omar Infante 2B 3.0 $9.2 $27.8 | 4 $7.6 $30.3 | 1.0 -$1.6 $2.4 8.8%
Scott Feldman SP 2.6 $8.8 $23.3 | 3 $10.0 $30.0 | 0.4 $1.2 $6.7 29.0%
Carlos Ruiz C 2.2 $8.4 $18.3 | 3 $8.7 $26.0 | 0.8 $0.3 $7.7 41.7%
Phil Hughes SP 2.4 $7.9 $18.7 | 3 $8.0 $24.0 | 0.6 $0.1 $5.3 28.1%
Bronson Arroyo SP 2.0 $8.3 $16.7 | 2 $11.8 $23.5 | 0.0 $3.4 $6.8 40.6%
Tim Hudson SP 1.6 $8.6 $13.9 | 2 $11.5 $23.0 | 0.4 $2.9 $9.1 65.5%
Scott Kazmir SP 2.3 $8.3 $18.7 | 2 $11.0 $22.0 | -0.3 $2.7 $3.3 17.4%
James Loney 1B 2.4 $7.6 $18.0 | 3 $7.0 $21.0 | 0.6 -$0.6 $3.0 16.4%
Jarrod Saltalamacchia C 3.6 $11.1 $40.3 | 3 $7.0 $21.0 | -0.6 -$4.1 -$19.3 -47.9%
Bartolo Colon SP 1.4 $8.3 $11.6 | 2 $10.0 $20.0 | 0.6 $1.7 $8.4 72.7%
Marlon Byrd COF 2.0 $7.5 $14.7 | 2 $8.0 $16.0 | 0.0 $0.5 $1.3 8.6%
Hiroki Kuroda SP 1.6 $14.8 $23.5 | 1 $16.0 $16.0 | -0.6 $1.2 -$7.5 -31.9%
A.J. Burnett SP 2.1 $13.1 $27.4 | 1 $16.0 $16.0 | -1.1 $2.9 -$11.4 -41.6%
Juan Uribe 3B 2.0 $8.1 $16.1 | 2 $7.5 $15.0 | 0.0 -$0.6 -$1.1 -6.6%
Ervin Santana SP 3.5 $13.3 $45.9 | 1 $14.1 $14.1 | -2.5 $0.8 -$31.8 -69.3%
David Murphy COF 1.9 $6.4 $11.8 | 2 $6.0 $12.0 | 0.1 -$0.4 $0.2 1.7%
Grant Balfour RP 2.2 $7.7 $16.9 | 2 $6.0 $12.0 | -0.2 -$1.7 -$4.9 -29.0%
Nate McLouth COF 2.1 $5.9 $12.2 | 2 $5.4 $10.8 | -0.1 -$0.6 -$1.4 -11.7%
David DeJesus COF 2.1 $6.2 $12.9 | 2 $5.3 $10.5 | -0.1 -$1.0 -$2.4 -18.5%
Stephen Drew SS 3.0 $11.0 $32.8 | 1 $10.1 $10.1 | -2.0 -$0.9 -$22.7 -69.3%
Dan Haren SP 1.8 $9.6 $16.8 | 1 $10.0 $10.0 | -0.8 $0.4 -$6.8 -40.6%
A.J. Pierzynski C 1.4 $7.6 $11.0 | 1 $8.3 $8.3 | -0.4 $0.6 -$2.7 -24.8%
Josh Johnson SP 1.6 $10.0 $16.3 | 1 $8.0 $8.0 | -0.6 -$2.0 -$8.3 -50.9%
Nelson Cruz COF 2.5 $10.6 $26.8 | 1 $8.0 $8.0 | -1.5 -$2.6 -$18.8 -70.1%
Kendrys Morales DH 2.5 $10.8 $27.3 | 1 $7.5 $7.5 | -1.5 -$3.3 -$19.8 -72.4%
Chris Young CF 2.0 $6.8 $13.9 | 1 $7.3 $7.3 | -1.0 $0.5 -$6.6 -47.8%
Adam Lind 1B 2.2 $7.0 $15.6 | 1 $7.0 $7.0 | -1.2 $0.0 -$8.6 -55.0%
Jason Hammel SP 2.1 $7.4 $15.5 | 1 $6.0 $6.0 | -1.1 -$1.4 -$9.5 -61.2%
Mark Ellis 2B 1.6 $5.5 $8.5 | 1 $5.3 $5.3 | -0.6 -$0.2 -$3.3 -38.3%
Roberto Hernandez SP 1.3 $3.6 $4.5 | 1 $4.5 $4.5 | -0.3 $0.9 $0.0 -0.4%
Gavin Floyd SP 1.6 $6.9 $10.8 | 1 $4.0 $4.0 | -0.6 -$2.9 -$6.8 -63.0%
Jesse Crain RP 2.1 $6.0 $12.2 | 1 $3.3 $3.3 | -1.1 -$2.7 -$9.0 -73.4%
Paul Maholm SP 2.1 $7.1 $14.8 | 1 $1.5 $1.5 | -1.1 -$5.6 -$13.3 -89.9%
                  AVG -0.2 $0.1 $0.3 -11.0%
                  RMSE 0.8 $1.7 $11.2 37.3%

A brief inspection of the table reveals two main observations:

• As has been the case in previous years, the crowd was generally conservative with regard to its estimates of the largest contracts. The offseason’s three most high-profile free agents — Robinson Cano, Jacoby Ellsbury, and Shin-Soo Choo — received ca. 40% more overall than the crowd expected (although the largest part of that figure is due to Choo’s contract, which was regarded as rather large at the time).

• Players received, on average, 11% fewer overall dollars than the crowd expected. Note, however, that merely adding 11% to every contract’s value doesn’t necessarily create a more accurate overall picture. In fact, subtracting 11% from every crowdsourced estimate actually leads to a larger RMSE (i.e. a larger average difference) between the crowdsourced and actual values. The reason: as noted above, the crowd’s misses weren’t evenly dispersed, but generally included underestimating the contracts given to more desirable free agents and, in many cases, overestimating the amount received by more ordinary free agents.

These points will likely be relevant to our newest data set, as well, when it’s released later this morning.





Carson Cistulli has published a book of aphorisms called Spirited Ejaculations of a New Enthusiast.

14 Comments
Oldest
Newest Most Voted
Jason B
12 years ago

Again, the dAAV column seems backwards; Cano made more than the crowd guessed, but you would think he made less. It’s especially confusing when the dYRS column shows it correctly (he got 2.4 more years than the crowd expected).

Jason B
12 years ago
Reply to  Jason B

Never mind, I’m a dumb ass. As you were. 🙂

Five Man Electrical Band
12 years ago
Reply to  Jason B

Do this, don’t do that. Can’t you read the sign?!

Well-Beered Englishman
12 years ago

Two new Banknotes Industries articles! NotGraphs lives on, friends; click my username for the link.

ljc
12 years ago

This post was interesting, but I can’t help think that every minute spent on a Fangraphs post today is a minute that could’ve been spent on a last-day Notgraphs post. Shame. #KeepNotGraphs #ButI’llTakeBanknotesIndustriesAsAConsolation

KCDaveInLA
12 years ago

Yay for this. #KeepBankNotesIndustries

Atreyu Jones
12 years ago

This whole series is interesting – thanks to the authors and contributors.

“As has been the case in previous years, the crowd was generally conservative with regard to its estimates of the largest contracts.”

And it happened yet again this year. It is virtually guaranteed that the Lester and Shields contracts will be bigger than the average/median ones listed in the crowdsourced results article.

DL80
12 years ago
Reply to  Atreyu Jones

It boggles the mind that GMs and owners haven’t yet realized that literally none of the huge 8+year megacontracts have worked out by the end.

I wonder how much is that GMs care about this year and next, and keeping their job, and know that it’s extremely unlikely they will still be around when it goes south anyway?

But shouldn’t owners then step in and be more proactive about signing (or not signing) these free agents?

Deelron
12 years ago
Reply to  DL80

I would posit that the Miguel Cabera contract signed in DET (the first one, running from 2008-2015 8/152) did work out, unless you argue that it worked out so well it in some way forced the second recent one.

DL80
12 years ago
Reply to  Deelron

Good call. 40.0 WAR just until 2014, not counting whatever he does next year. Not sure what the value per win was in 2008, but even if it was as low as $4 mil/win (or even a little lower), it’ll work out. The next one, though…

TKDC
12 years ago

Interesting, by percentage, Bartolo Colon was the most “overpaid” and the team was right/crowd was wrong and Paul Maholm was the most underpaid and again the team was right/crowd was wrong.

Psy Jung
12 years ago

Interesting about the systematic tendencies – I wonder if that mean that teams believe that the the value of a win isn’t linear. I guess it could also mean that teams are still bad at free agency, though.

TKDC
12 years ago
Reply to  Psy Jung

Or that the crowd doesn’t know how to predict wins for better players?

Hank
12 years ago

Carson – real good stuff. Always good to go back and look at past projections/crowsourcing.

One nitpick/suggestion – this might be something that would be better presented graphically as it may be a bit more clear on the trends (variance based on contract size or contract years, where people are getting it wrong more of the time, etc)

Of course that’s easy for me to say as I’m not doing the work, but perhaps next time?