Projections Differences Part I: Hitters

This is the time of year I begin my annual ritual of collecting and merging together various player projection systems in preparation for my Scoresheet draft. This is often a long and arduous process that involves lots of merging on string variables — thanks to no common id — and lots of data cleaning. Thankfully, FanGraphs provides a number of projection systems free of charge on the website that are both exportable and available with a common id number!

In this post I’ll focus on three projection systems — Steamer, Marcel, ZiPS — to see if there are significant differences in how they project various players to perform in 2012. Read this post by Matt Schwartz to see how this systems performed in the aggregate for 2011 and be sure to visit the websites of the authors of these systems to learn more about the details of how they are estimated.

The first thing to note about these systems is that they vary widely in playing time assumptions. In the common sample of 458 hitters, ZiPS projects a mean number of at-bats of 439, Marcel comes in at 360, while Steamer comes in at 339. As the table below shows, the correlations among the systems are not particularly high either. These differences in playing time will produce vast differences in counting stat projections, so we are going to ignore those and focus instead of rate stats.

At-bats
	Marcel	Steamer	ZiPS
Marcel	1
Steamer	0.76	1
ZiPS	0.41	0.5	1

Turning to rate stats in the aggregate, we see a lot more similarity. The tables below report correlations for OPS and wOBA for all three projection systems. All three systems have very high correlations with the other systems. Steamer and ZiPS correlate higher with each other than either of them does with Marcel, which is likely due to the intended naiveté of the Marcel system.

OPS
	Marcel	Steamer	ZiPS
Marcel	1
Steamer	0.87	1
ZiPS	0.86	0.94	1

wOBA
	Marcel	Steamer	ZiPS
Marcel	1
Steamer	0.86	1
ZiPS	0.85	0.94	1

The differences among the systems get more interesting if we focus on individual players. The table below lists 2012 OPS predictions for the 15 position players who have the highest projected variance in their the OPS. To restrict this post to mostly everyday players, I set a minimum of 300 projected at-bats. In addition to the projections, the table presents 2011 OPS and career OPS for each of these 15 players.

This is quite a diverse group. The list is heavy on young players with either limited MLB playing time (Weeks, Mayberry, Guzman) or large variance in past performance (Heyward, Alvarez, Snider). There are two run of the mill veteran catchers (Olivo, Barajas), several players who are changing teams (Cuddyer, Reddick, Smith, Pujols), and two of the best players in baseball (Longoria, Pujols).

Name	Marcel OPS	Steamer OPS	ZiPS OPS	2011 OPS	Career OPS
Josh Reddick	0.749	0.764	0.680	0.784	0.706
Jemile Weeks	0.770	0.695	0.692	0.761	0.761
Travis Snider	0.729	0.788	0.711	0.617	0.730
Wilson Valdez	0.659	0.584	0.630	0.630	0.620
John Mayberry	0.797	0.746	0.727	0.854	0.846
Michael Cuddyer	0.769	0.815	0.837	0.805	0.794
Jesus Guzman	0.790	0.726	0.723	0.847	0.822
Jason Heyward	0.802	0.853	0.787	0.708	0.789
Rod Barajas	0.684	0.665	0.731	0.717	0.698
Albert Pujols	0.933	0.999	0.952	0.907	1.037
Seth Smith	0.795	0.762	0.730	0.830	0.833
Evan Longoria	0.846	0.910	0.881	0.850	0.875
Miguel Olivo	0.701	0.650	0.639	0.641	0.700
Giancarlo Stanton	0.873	0.935	0.910	0.893	0.869
Pedro Alvarez	0.712	0.773	0.770	0.561	0.696

Are there common factors that appear to lead to divergent projections? The short answer is yes. Young players with either limited experience or divergent previous results and players changing teams make up the bulk of this list. The perfect storm for projection systems is Josh Reddick. Reddick made the best of his first shot at significant playing time last year posting a .784 OPS in 278 plate appearances. Marcel and Steamer see his 2011 line as a good model for his 2012, while ZiPS is quite negative on Reddick, with a .680 projected line. Marcel and Steamer buy into Reddick’s 2011 performance, while ZiPS uses Reddick’s poor 2009 and 2010 lines in limited playing time to hold down his forecast. Moving to Oakland will not help Reddick’s power numbers, but the systems disagree most on his OBP, with ZiPS projecting the lowest walk rate and BABIP for Reddick.

With the exception of Jemile Weeks, Steamer is the most optimistic about the younger players on this list. Steamer sees Travis Snider producing at a level that most have projected for years. Steamer also projects that Jason Heyward will revert to 2010 form, that Pedro Alvarez will bounce back, and sees increased production for Giancarlo Stanton. Steamer’s optimism about young players is consistent with Matt Schwartz’s work referenced above, which suggested that Steamer projections tended to be riskier in nature.

Two of the surprising names on this list are Albert Pujols and Evan Longoria. Both are established stars with consistent past production. Steamer sees a career-best year for Longoria in his age 26 season, with no lingering effects of his 2011 injuries, which is not a bad guess at all considering that Longoria should be entering the peak production phase of his career. This is where Marcel’s simplicity works may lead to it being too conservative as it project a career worst year for Longoria, which is not the typical production pattern for a star-level player entering his prime.

Steamer is the most optimistic system for Albert Pujols as well. All three of these systems see him rebounding from a 2011 in which he posted his first ever season with a wOBA less than .400, but only Steamer sees him back above .400 in 2012. Steamer essentially sees vintage 2010 Pujols showing up this year, while Marcel and ZiPS see his declining OBP and power continuing unabated. In general I’d be inclined to side with Marcels and ZiPS in this case as players on the wrong side of 30 are not often able to avert the decline phase of their careers, but Pujols is not your normal 32 year old hitter. Given his new contract it is clear that the Angels fans hope that Steamer is correct.

The bottom line on projections for hitters is that there is not a lot of variance to discuss. The median variance in OPS for this sample is only .025, which is not enough to get overly excited about. The 15 guys in the table above standout, but even for these guys the prediction variance is not enough to fundamentally change how we view a player. Pujols will be a lineup force whether his OPS is closer to .930 or closer to 1.000, while Phillies fans will continue to hope that Wilson Valdez does not get a lot of playing whether he has a .580 or a .650 OPS. Where the systems do differ quite a bit is in their pitching projections, which will be covered next week.