10 Years of Team Performance, 10 Years of Team Projections

I’ve got a post going up at JABO today. It might’ve already gone up, I don’t know — I’m writing this in advance, and I wrote that post in advance, and I’m not in charge of when they publish the content that’s been sent their way. But today, there is this post and that JABO post, and within the latter, you will see the following image:

actualprojectedwins_2005_2014

You’re smart enough to get this. That’s a whole decade of information, showing single-season projected team win totals, and single-season actual team win totals. You observe both a relationship, and a fair amount of noise. I suppose I don’t need to spoil that other post. Read that other post, whenever it’s published. Read that other post twice! Click click click!

Yet seldom do we dig into historical team projections. There are reasons for this, but now that I’ve gone to the trouble of gathering all the necessary data, there’s some other stuff we can do. There’s a lot of other stuff we can do; I have done a small amount of it.

As in the JABO post, I need to acknowledge the issues, here. The bulk of the information was supplied by Replacement Level Yankees Weblog, which has a history of preseason team-projection blowouts. They date back to 2005, and they blend multiple projection systems, through a simulator. ZiPS has been included, and PECOTA has been included, and so on and so forth. But between some years, some different systems have been selected. The systems themselves have changed as we’ve learned more and more about how baseball and forecasting work. It stands to reason that the projections will be worse the further back you go.

But I think it’s still OK to proceed. You see the relationship in the first image. And, really, projections haven’t changed that much over the years. They’re still pretty good, and they’re still all imperfect. Systems haven’t improved by leaps and bounds. One system will mostly agree with another system, such that you don’t usually see one team being either good or bad based on two different forecasts. There are a lot of projection systems. They do basically the same thing, using basically the same numbers.

We can accept there are some issues and still play with the data. There are some issues with WAR, but that doesn’t stop us from playing with WAR. So, acknowledging the issues, let’s get to the fun stuff. The first thing that occurred to me: compare team performance to projected team performance over the decade. So, over a combined 10 years, which teams exceeded their projections the most, and which teams most dramatically fell short?

actualprojected20052014

Leading the way are the Angels, who, since 2005, have topped their projected win total by 47, or nearly five a year. Right behind the Angels are the White Sox, at 43, and Dave actually wrote about this two years ago. He gave a lot of credit to Don Cooper helping to keep starting pitchers healthy. There’s a pretty substantial gap between the second-place White Sox and the third-place Tigers.

One standard deviation here is 21. There are six teams who fall outside of that range, and the most extreme data point is that of the Cubs. The Cubs show up at -55; next-worst are the Mariners, who are better by 19. I don’t know exactly how to explain this, but what’s happened has happened, so if you’ve felt like the Cubs have mostly been underachievers, this is support for that. The Mariners, incidentally, are mostly where they are because of the crippling disappointments of 2008 and 2010. That the Red Sox and Rockies also show up pretty strongly negative makes you wonder about potential park effects. I don’t know, it’s complicated. I’m just trying to show you stuff, not necessarily explain it.

That looked at the last 10 years. What if we just looked at the last five years?

actualprojected20102014

Now the Orioles show up. And while the Orioles are at +26 wins over five years, they’re actually at +44 wins over three years. There’s been nothing quite like their 2012. Of some interest, look at the Angels and White Sox. They were the leaders in the previous graph, but here they fall way back to the pack. The White Sox are actually at -1. They did their projection-beating years and years ago, for the most part.

At the bad end, some familiar names. The Rockies have badly underachieved. The Cubs’ pattern hasn’t disappeared. The Red Sox, of course, have been somewhat bipolar, and as for the Marlins, they were a mess, relative to expectations, in 2011 and 2012. The Marlins had a particularly fun five-year stretch:

  • 2008: +16 wins
  • 2009: +15
  • 2010: 0
  • 2011: -10
  • 2012: -15

Looking at just the last three years — I didn’t prepare an image, but, the leaders are the Orioles, A’s, and Pirates. Bringing up the rear: the Rockies, Red Sox, and Blue Jays. You can definitely see the effect health has, and over smaller samples, that doesn’t get much of an opportunity to balance out.

Real quick, before I go, let’s take a look at sustainability. If you beat a projection in Year X, does that mean anything for Year X+1? If you fall short of a projection in Year X, does that mean anything for Year X+1? Spoiler alert: no, basically. On the x-axis: actual wins – projected wins, Year X. On the y-axis: actual wins – projected wins, Year X+1. I don’t think I need to show you the r-squared value.

projectedactualwins_yearx_yearx1

Take all the teams that beat their projections by at least 10 wins. In Year X, they averaged a difference of +14 wins. In Year X+1, they averaged a difference of 0 wins. Now take all the teams that fell short of their projections by at least 10 wins. In Year X, they averaged a difference of -15 wins. In Year X+1, they averaged a difference of 0 wins. It’s not conclusive, but I don’t know what else to tell you. The evidence suggests this is magic, and baseball teams aren’t magical.

Of course, this probably comes off like a shot at the Orioles. It’s not intended that way; I’m just following the numbers. Because the Orioles have been better than expected for three years in a row, for the most part, consensus is mounting that there’s just something special about them. And I’m totally willing to concede that it’s possible, but it doesn’t have to be 0% signal or 100% signal. It can be signal and noise, and I don’t think the Orioles should bet on consistently overachieving by several wins. In recent history, it’s been a difficult thing to pull off.

The Angels did it, right until they stopped. The Giants have done it, but not in the odd years. The Marlins had those back-to-back years I showed you above. Take the 2008 – 2010 Twins: they overachieved by a combined 31 wins, then they underachieved by 21 in 2011 alone. I’ll remind you of the earlier White Sox example. This isn’t me being stubborn; this is me just not finding enough precedent. It’s difficult to believe the Orioles have just figured something amazing out. It’s not impossible, but they might well be the first.

Anyway, I think I’ve written enough. That’s 10 years of team performance, and 10 years of team projections. I’m not sure how much we’ve learned, but we’ve at least validated the feelings of Cubs fans. Sorry, or, you’re welcome.





Jeff made Lookout Landing a thing, but he does not still write there about the Mariners. He does write here, sometimes about the Mariners, but usually not.

52 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Jaack
9 years ago

But I thought that fangraphs hated the Orioles irrationally. Have the comment sections been lying to me?