Yes, Preseason Projections Still Matter

Baseball fans are often unhappy with projections, and they seem to enjoy sharing that unhappiness with me. That’s hardly surprising, of course — projections have large error bars (which we try to express when discussing them), and on some level, they represent a computer saying mean things about players you might like. You probably aren’t shocked by this, though you might be surprised to learn that the time of year I get the most complaints is during the stretch run rather than the preseason. And while some of the complaints I hear are about projections that missed the mark, the majority are about how little the in-season projections have changed since the start of the year. It can be frustrating when a player who is slugging .500 despite a preseason SLG projection of .350 only has a rest-of-season SLG projection of .400. Clearly, the conventional wisdom says, the projections are too slow to change.
Ben Clemens has written a number of pieces auditing our playoff odds, and while they aren’t as meticulously perfected as a 1996 Lexus, they’re pretty darn good. These calculations rely heavily on our in-season projections, which at their gooey, caramel center are an attempt to combine what we knew about a player or team before the season (in the form of preseason projections) with what we’ve learned since.
The relevant question then becomes how much a few months of performance should change what we thought about a player in March. And as it turns out, it’s less than the folks who find projections frustrating might think. When it comes to ZiPS, the preseason projections, both for teams and individual players, are more predictive of rest-of-season performance than season-to-date numbers. That holds true even very late in the season, when there’s been a lot of cumulative in-season performance to look at. I’ve tested preseason projections versus in-season performance as late as September 10 each year, and the preseason projections never clearly do worse than the in-season stats. Since we’re right at the start of September, I thought this would be a good opportunity to look at how the ZiPS preseason projections stack up against March-through-August performance when it comes to projecting what happens in the final month of the season.
And to be clear, I don’t think this preseason edge is exclusive to ZiPS; I expect that Steamer and PECOTA perform similarly well. It’s just that I have a rather unique level of access to ZiPS. But data is more important than a simple claim, so let’s dig in. Read the rest of this entry »







