Where ZIPS and Steamer Disagree

With the ZIPS projections now loaded onto the site and the player pages, I thought it’d be fun to take a look at a few examples where ZIPS and Steamer — probably my two preferred projection systems at the moment — differ this year, and whether there’s anything in particular we can learn from those differences.

First, I wanted to whittle down the population of players that I was dealing with. While I appreciate Steamer’s pluckiness, 4,136 projections for position players might be a little bit of overkill. I, for one, am not overly concerned with how Jeyckol De Leon is going to perform this year. Maybe it’s just me.

ZIPS projects a slightly more sane number of position players — 1,046, to be exact — but even that is still a little unwieldy, as a good chunk of those guys are sub-replacement level minor leaguers who aren’t going to see the Majors this year. By and large, we care mostly about the projections for players who are going to see substantial regular playing time in the big leagues this year, or at least, I do. Carson can care about all the fringe prospects he wants; I’ll leave that to him.

So, in order to get a list of projections for guys we care about, I excluded players who had never been in the majors or were projected to be below replacement level, leaving us with 601 Major League position players. That was still a little unwieldy, though, so I took the top 180 players by the average WAR of the two systems, which gave us a good selection of players that are projected to be league average or better by one of the two systems.

From there, it was a pretty simple sorting task to find some big differences, but many of them are driven by assumptions about playing time rather than big gaps in the actual projections. For instance, ZIPS lists Brett Jackson as a +2.5 WAR player, while Steamer comes in at just +0.9, which seems like a huge difference of opinion until you realize that the ZIPS number is based on 626 PA and the Steamer number is based on 274 PA. Rescale them both to 600 PA, and the gap is just 2.4/2.0, showing that the two systems are essentially in agreement on Jackson’s overall profile for 2013.

Since I care more about the differences in the projected performances than in the playing time gaps, we’ll focus on the big discrepancies in projections per 600 plate appearances (or 450 PA for catchers). Putting things on the same playing time scale will inflate the numbers of part-time players and injured guys while depressing the numbers of durable iron men, but we’ve already gotten rid of most of the part-time guys with our filtering, so that’s a trade-off I’m willing to make. Oh, and thankfully, the two systems were already on almost identical scales for these players, so there was no need to make any adjustments to the numbers as we did with the Fans projections last week.

Long introduction finished… on to the differences. We’ll start with the 14 players where Steamer projects at least +1 additional WAR per full season (600 PA non-catchers, 450 PA catchers):

Name ZIPS WAR ZIPS WAR/600 Steamer WAR Steamer WAR/600 Difference
Justin Upton 3.3 3.0 5.1 4.7 1.7
Pablo Sandoval 3.4 3.7 4.5 5.0 1.3
Daniel Murphy 1.0 1.3 2.6 2.5 1.2
Wilin Rosario 2.6 2.5 3.5 3.7 1.1
David DeJesus 1.4 1.7 2.6 2.7 1.1
Michael Young 1.3 1.3 2.2 2.3 1.1
Paul Goldschmidt 2.5 2.3 3.3 3.3 1.0
Nick Swisher 2.4 2.4 3.5 3.4 1.0
Alex Avila 3.1 3.0 3.5 3.9 1.0
Allen Craig 1.9 2.3 3.0 3.3 1.0
Lorenzo Cain 1.5 1.9 2.3 2.9 1.0
David Wright 3.8 3.7 5.1 4.7 1.0
Jesus Montero 1.9 1.4 2.6 2.4 1.0

While it was pretty easy to spot patterns with the differences between the fans and Steamer, this is a pretty interesting variety of player types. There’s no regular in baseball that Steamer likes more relative to ZIPS than Justin Upton, as Steamer is projecting a .374 wOBA while ZIPS comes in at just .342. Steamer projects six more doubles, six more home runs, and 22 fewer strikeouts in nearly equal numbers of playing time, so this simply comes down to a difference in opinion on how well Upton’s performance will translate outside of Arizona.

The three guys behind Upton are all in their mid-20s and theoretically headed towards their primes, so if you stopped after the top four, you might assume that Steamer is more optimistic about young players improving than ZIPS is. However, you’ll see shortly that we can reject that hypothesis pretty easily, and after all, these four are followed by the likes of David DeJesus (33), Michael Young (36), and Nick Swisher (32), so there doesn’t appear to be an obvious biased towards a particular age range here.

If you squint, you can kind of see a pattern towards favoring guys who hit for power, but you have to ignore Cain, Murphy, DeJesus, and Young in order to see that pattern really stick. In reality, it seems like this is just a pretty big potpourri of player types, and Steamer has differing reasons for preferring them. With all the variables that go into projections — aging curves, park effects, differing amounts of regression on certain numbers — it’s natural that there are going to be places where two well constructed models diverge. The fact that we can’t easily identify a common reason for the difference might actually be considered a good thing.

Let’s move on and look at the players that ZIPS likes more than Steamer. Again, we’re looking at players where’s at least a +1 WAR difference per 600 PA (or 450 PA for catchers).

Name ZIPS WAR ZIPS WAR/600 Steamer WAR Steamer WAR/600 Difference
Jurickson Profar 3.7 3.6 0.7 1.2 2.4
Bryce Harper 4.7 4.4 3.0 2.7 1.7
Craig Gentry 2.5 4.5 1.1 2.8 1.7
David Ortiz 3.1 4.4 2.4 2.9 1.6
Giancarlo Stanton 6.4 6.7 5.6 5.5 1.3
Mike Olt 2.9 3.6 0.7 2.5 1.1
Brett Lawrie 4.5 4.7 3.4 3.7 1.0
Torii Hunter 2.8 3.0 1.8 2.0 1.0

And here’s why we can discard the notion that Steamer likes young players more than ZIPS. Of the eight players where ZIPS is more bullish, two of them are 20-year-olds, two others are 23-year-olds, and there’s a 24-year-old. And then there’s two 37-year-olds, just for fun. So, ZIPS likes both the very young and the very old more than Steamer. Or, at least, a couple of very young and a couple of very old players. With just 22 players where there’s a gap of +1 WAR per full season, we can’t really draw any hugely firm conclusions about the differences.

I do think it’s interesting that ZIPS goes along with the fans optimism about Harper and Profar, however. We noted that the fans were far more optimistic than Steamer in regards to players with very short Major League track records, but ZIPS is actually quite a bit more bullish on these two as well. To see if this was a persistent pattern, I went and looked at the projections for players who had never played an MLB game, and ZIPS was consistently higher for most of them as well. There are 15 minor leaguers that ZIPS projects out to +2 WAR/600 or better for 2013, and ZIPS is higher on every single one of those 15 than Steamer is.

So, between ZIPS, Steamer, and the Fans, it seems like Steamer is at the low end on prospects and very young players, ZIPS rests in the middle, and the Fans are at the high end.

Of course, none of that explains why ZIPS is also higher on David Ortiz and Torii Hunter than Steamer, but it could be that Dan Szymborski is using a flatter aging curve than the guys calculating Steamer are. That’s just a wild guess based on two data points, though, so don’t take that guess seriously. As with before, it’s informational to know where the two systems differ, but don’t draw any conclusions about whether one is more right than the other based on these examples.

Many tests have shown that combining multiple projections is better than simply relying on one system anyway, so that methodology will smooth many of these gaps. If one good projection system thinks Upton is a +3 win player and another thinks he’s a +5 win player, then I’d be just fine assuming that he’s actually a +4 win player. The truth often lies in the middle ground. I suspect we might find that these players end up not too far from the number between their ZIPS and Steamer projections for 2013.





Dave is the Managing Editor of FanGraphs.

24 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Kevin
11 years ago

Great stuff. Will there be a similar article on pitchers?

Hmm
11 years ago
Reply to  Dave Cameron

On pitchers, doesn’t Steamer factor in fastball velocity? I suspect that will lead to some substantive (but explainable) differences based on methodology.