Looking at the 2010 Fan Projections: Part 2

by
January 4, 2011

Yesterday I looked at the 2010 Fan Projections for position players, and specifically how much higher fans of a team projected players on that team compared to non-fans. It turned out to be by about half a win. Commenters to that post wondered which group did a better job projecting the actual performance of the players.

Tango found that the Fan Projections were in the middle of the pack compared to other projection systems: a respectable 10 out of 21, up against the big hitters like CHONE, CAIRO, and Bloomberg. But that was with the Fans as a whole, not split out by the fans’ favorite teams.

It turns out that the fans let their optimism get in the way and did a poorer job projecting players on their favorite team. The root mean square error (lower is more accurate) comparing the projected WAR to actual WAR was 2.1 for projections by favorite-team fans compared to 1.8 for all other fans. A more intuitive way of thinking about it is the mean absolute error — that is the average each projection system was off across all players. For favorite-team fans this was 1.7 wins versus 1.5 wins for all other fans — a difference of about two runs. So, overall it is not a huge difference.

Both groups were overly optimistic: 73% of players underperformed their favorite-team-fan projection and 64% their rest-of-fans projection. The favorite-team fans were, on average, one win high while the non-fans half a win high. Here is the actual WAR for the 206 position players plotted against the WAR projected by fans of that player’s team and WAR projected by everyone else. Again, the red line is when the two are equal. Dots above the line represent players who over-performed their projection; and those below, under.

You can see that the graph to the left has a great ‘spread’ from the line — that is, the poorer projection by favorite-team fans. The amount by which the ‘blob’ of points is centered below the lines shows the under-performance (or over-projection). You can see that both are centered below the line, but that the one to the left more so.

Quick note: it looks like there are not nearly as many projections for pitchers, so I am not able to run these same type of comparisons for them. With position players, there were 206 with at least 10 projections by fans of the team. But for pitchers there were just five players with 10 and just 25 with at least five projections. I am not sure why there are so many fewer pitcher projections.

You Aren't a FanGraphs Member
It looks like you aren't yet a FanGraphs Member (or aren't logged in). We aren't mad, just disappointed.
We get it. You want to read this article. But before we let you get back to it, we'd like to point out a few of the good reasons why you should become a Member.
1. Ad Free viewing! We won't bug you with this ad, or any other.
2. Unlimited articles! Non-Members only get to read 10 free articles a month. Members never get cut off.
3. Dark mode and Classic mode!
4. Custom player page dashboards! Choose the player cards you want, in the order you want them.
5. One-click data exports! Export our projections and leaderboards for your personal projects.
6. Remove the photos on the home page! (Honestly, this doesn't sound so great to us, but some people wanted it, and we like to give our Members what they want.)
7. Even more Steamer projections! We have handedness, percentile, and context neutral projections available for Members only.
8. Get FanGraphs Walk-Off, a customized year end review! Find out exactly how you used FanGraphs this year, and how that compares to other Members. Don't be a victim of FOMO.
9. A weekly mailbag column, exclusively for Members.
10. Help support FanGraphs and our entire staff! Our Members provide us with critical resources to improve the site and deliver new features!
We hope you'll consider a Membership today, for yourself or as a gift! And we realize this has been an awfully long sales pitch, so we've also removed all the other ads in this article. We didn't want to overdo it.




 

Dave Allen's other baseball work can be found at Baseball Analysts.

17 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
T
14 years ago

One small thought – There are a lot of Mariner fans on this blog, and almost every player on that team had an unprecedented collapse. Any possibility that skewed the results at all?

4