The Projections In Review, Briefly
You might’ve noticed that the regular season is over. As such, all the regular-season numbers are in the books, which gives us some good opportunities for evaluation on the outside. In this quick post, I would like to evaluate the preseason team projections. Last year, at least in the American League, the projections wound up being a mess. I remember there being a point about halfway through where it looked like one would’ve been smarter to bet the opposite of every AL team projection. How’d the numbers shake out in 2016? Below, see plots.
Several times before, I’ve used old team projections from seasons past. Not all team projections included here come from the same sources, because the same sources just haven’t always existed. For recent years, I’ve been able to use FanGraphs team projections. Going further back, I’ve had to search elsewhere, because FanGraphs just didn’t have projections. So I know that’s one potential source of error here, but I think it’s better than just not having data at all. And ultimately, all projection systems are built around similar foundations. You take recent numbers and weigh them and project them for the short-term future. There’s not a lot to change. So! Why don’t we just get to the information?
I have team projections stretching back to 2005. Here is a plot including actual team wins vs. projected team wins. What you see here is the average error per team per season:
This year, the projections fared much better than they did in 2015. I went with the last version of FanGraphs’ preseason team projections, and after the average projection missed by 8.1 wins a year ago, this time the average projection missed by 5.7 wins. That puts this year in line with 2014, as being fairly successful, math-wise. It would be the best-projected year since 2007. I have no idea if that means anything; I’m just putting it out there. This year’s biggest miss was the Twins, who fell an incredible 19 wins short of the March expectation. The word “Twins” has the word “wins” right in it. It also has a T, which looks like the symbol for perpendicularity. Other teams might be content to operate in parallel with winning. The Twins decided to challenge it head-on.
Another thing we can look at: What about BaseRuns wins vs. projected wins? We know there are elements that are just about un-projectable. What if we strip those away?
Last year, the average miss was 7.0 BaseRuns wins. This year, the average miss was 5.4 BaseRuns wins, standing again as the strongest year for the projections since 2007. You might say it’s strange that the projections haven’t improved on a set of projections from a whole decade ago, since that was so far back it was pre-PITCHf/x era. But this is at least evidence that last year’s weirdness was a blip. The biggest miss for 2016: The Red Sox, actually. They were projected for 89 wins, but they finished with 102 BaseRuns wins. Good team, the Red Sox. The Cubs were the second-biggest miss!
Just to close it out, we can leave out the projections entirely. Here are BaseRuns wins vs. actual wins. Are teams finding ways to beat their underlying statistics more often, or is that not the case?
This year, the average difference was 4.2 wins. That’s down from last year’s 5.1, but still, this is the second-biggest error in the sample. So that’s potentially of note. Last year it felt like the BaseRuns model was practically broken. This year has eased some of those concerns, but it’s still worth wondering about. The Rangers finished with 13 more actual wins than BaseRuns wins. The Rays finished with 13 fewer actual wins than BaseRuns wins. Go ahead and figure that one out. At least for the time being, I quit. What’s done is done!
Jeff made Lookout Landing a thing, but he does not still write there about the Mariners. He does write here, sometimes about the Mariners, but usually not.
<3