Looking at the 2010 Fan Projections: Part 2

Yesterday I looked at the 2010 Fan Projections for position players, and specifically how much higher fans of a team projected players on that team compared to non-fans. It turned out to be by about half a win. Commenters to that post wondered which group did a better job projecting the actual performance of the players.

Tango found that the Fan Projections were in the middle of the pack compared to other projection systems: a respectable 10 out of 21, up against the big hitters like CHONE, CAIRO, and Bloomberg. But that was with the Fans as a whole, not split out by the fans’ favorite teams.

It turns out that the fans let their optimism get in the way and did a poorer job projecting players on their favorite team. The root mean square error (lower is more accurate) comparing the projected WAR to actual WAR was 2.1 for projections by favorite-team fans compared to 1.8 for all other fans. A more intuitive way of thinking about it is the mean absolute error — that is the average each projection system was off across all players. For favorite-team fans this was 1.7 wins versus 1.5 wins for all other fans — a difference of about two runs. So, overall it is not a huge difference.

Both groups were overly optimistic: 73% of players underperformed their favorite-team-fan projection and 64% their rest-of-fans projection. The favorite-team fans were, on average, one win high while the non-fans half a win high. Here is the actual WAR for the 206 position players plotted against the WAR projected by fans of that player’s team and WAR projected by everyone else. Again, the red line is when the two are equal. Dots above the line represent players who over-performed their projection; and those below, under.

You can see that the graph to the left has a great ‘spread’ from the line — that is, the poorer projection by favorite-team fans. The amount by which the ‘blob’ of points is centered below the lines shows the under-performance (or over-projection). You can see that both are centered below the line, but that the one to the left more so.

Quick note: it looks like there are not nearly as many projections for pitchers, so I am not able to run these same type of comparisons for them. With position players, there were 206 with at least 10 projections by fans of the team. But for pitchers there were just five players with 10 and just 25 with at least five projections. I am not sure why there are so many fewer pitcher projections.