Wisdom Of The Crowd

by
November 30, 2009

Today, David announced the newest addition to the site, and one we’re all pretty excited about it – Fan Projections. We’ve hosted the forecasts of most of the various top projection systems over the last few years, and you’ve probably become accustomed to hearing various writers quote CHONE, ZiPS, or Marcel. With FanGraphs now offering the ability to aggregate projections from various sources, we’ll have a new set of projections to offer this winter – those of the crowd.

If you’ve read James Surowiecki’s “The Wisdom of Crowds”, you’ll no doubt recognize the theory behind the endeavor. As Surowiecki suggests, there is evidence that certain groups of lay people can give better estimates than any single expert, due to the unique experiences we all have in life. By blending our understandings together, we can eliminate some of our individual biases and enhance the shared wisdom of the population.

Tom Tango has done some research on this as it pertains to projections in baseball, and Surowiecki’s theory holds up pretty well. In a four part series that matched up six projection systems against 165 fans, the aggregate projections of the fans was essentially the equal of the complex statistical models. Individual fans by themselves didn’t fare so well, but when all fans were combined into a single projection, they held their own.

This is, essentially, Surowiecki’s argument in a nutshell. We all have our limitations of understanding, but the whole can be greater than the sum of its parts.

So, in that spirit, we offer you the opportunity to make your voice heard. Use the Fan Projections to add your personal wisdom the crowd, based on your insights and experiences. Despite the fact that this is site is statistically inclined, we have a fairly broad base of readers, offering a wide variety of opinions and views – the kind of crowd where the wisdom of many can really shine through. Try projecting some players every day, putting real thought into what you expect from each player in 2010.

If we have the diverse, intelligent crowd that I think reads this site, don’t be too surprised if the Fan Projections end up hanging with the big boys. Let’s put the wisdom of the FanGraphs crowd to use and see what happens.





 

Dave is the Managing Editor of FanGraphs.

21 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
JD Sussman
14 years ago

Dave,

I have not read “Wisdom of the Crowds” but is that because the fans tend to have a natural regression in place that is similar to those in the statistical models?

0
JoeR43
14 years ago
Reply to  JD Sussman

It’s just essentially the theory that when enough people use simple past performance and observation to judge a player, eventually you get a real consensus on said player.

And that a sample size of just 165 people can come to about the same conclusion as complex analytical processes, it shows the power of simple objectivity.

0
philosofoolmember
14 years ago
Reply to  JD Sussman

The phenomenon of the wisdom of the crowds is well known but not well understood. There are many hypotheses about why it works and none are perfect. Some people offer a “cancelling out” explanation: the idea is that our personal biases each other out (for every pessimist, there’s an optimist). I find this explanation partial at best, since it leaves out why the remainder after all our biases cancel tends toward reality.

Another explanation appeals to the Condorcet Jury Theorem, which is just a theorem of probability mathematics; it says that given a set of independent trials with a greater than .5 chance of getting the right answer to a yes/no question, the probability that the majority of trials is correct gets arbitrarily close to 1 as the number of trials approaches infinity. (There are ways to show that any complex question that isn’t a yes/no question can be reduced to a series of yes/no questions, so the theorem can be applied to a complex question like “How many home runs will Jeter hit?”)

There are cases where the Wisdom goes completely unwise. Here’s a fun example: if you ask a room full of people to secretly write the number of beans in a jar on a piece of paper, the average answer tends to be very close to the number of beans in the jar. If instead you ask each person, in sequence, to say out loud how many they think are in the jar so everyone hears everyone else’s answer, the average answer usually converges on the first answer that everyone heard. Crowds can be very stupid and sheepish under certain conditions. An explanation of this sort is apt to make the fans scouting report a lot like projection systems because many people will simply look at a few projection systems before offering their forecast. What is interesting is whether enough fans can beat the projection systems by knowing things that the systems don’t, e.g., injury history.

4
JD Sussman
14 years ago
Reply to  philosofool

Interesting stuff. Thanks.

0
walkoffblast
14 years ago
Reply to  philosofool

It was interesting to watch what you are talking about in the last paragraph play out favorably with Predictatron. It will likely be even more interesting to see how this experiment plays out.

0
Toffer Peak
14 years ago
Reply to  JD Sussman

The Wikipedia Article has a surprisingly good break down of the phenomenon and the criteria necessary for it to be successful. – http://en.wikipedia.org/wiki/The_Wisdom_of_Crowds

My biggest concern would be bias (Independence). Since this site already lists Bill James’ projections, the fan’s and in the following weeks, CHONE, ZiPS, etc. a lot of fans will probably be biased by those numbers. To be most effective WOTC predictions need to be “blind”. Unfortunately I don’t think there is any way to realistically fix this. Fortunately those projection systems are pretty good so this variable shouldn’t too negatively affect the results.

The other concern I have that has been mentioned is ensuring that everyone has good intentions. On the Internet their are always jerks who are willing to ruin a great thing. An “easy” way to correct this would be to put your money where your mouth is. Submissions could cost say $.25-$1 with the person whose projection was closest at the end of the year getting all of the winnings. This would incentivize people to only submit their true beliefs. Of course this would severely reduce the number of submissions as well as be a huge burden on Fangraphs to create such a system.

Overall I think this is an awesome idea (something that I unfortunately had been hoping to implement myself in the future) and as long as they can control for crazy submissions I think the final results should be quite accurate.

0
philosofoolmember
14 years ago
Reply to  Toffer Peak

When it comes to jerks, the best you can hope for is that jerks washout by contradicting one another or simply get swamped by the vast majority of people that are just giving their best opinion.

0