The Cubs Vs. a Decade of Projections

Earlier, I published a post including this graph:

actualprojected20052014

Of all the things that stand out, perhaps nothing stands out quite like the Cubs. Over the last 10 years, they’ve won 55 fewer games than they’ve been projected to win, and while there are some quibbles you can have with blending all the years and all the projection systems, at the end of the day, projections aren’t too dissimilar and they haven’t changed super dramatically over the decade, and no one comes particularly close to matching the Cubs’ level of disappointment. At -55, they’re separated from the next-lowest team by 19 wins.

I don’t think there’s really anything predictive, here. I’d never bet on a given team beating or undershooting its next-year projection. Certainly not based on what it just did. But -55 seems worthy of investigation. What can we find out, about why this might’ve happened? What traits or events might’ve contributed to the Cubs coming off as such a disappointment?

To begin with, an obvious step. Over the last 10 years, the Cubs have won 756 games. But it’s worth checking on their run differentials, to see if everything matches up. And as it turns out, the Cubs have undershot their own Pythagorean records by a little bit. According to the runs scored and runs allowed, then over the decade, the Cubs should’ve won 769 games, which is an increase of 13. Maybe you don’t like the use of the word “should’ve”, but you know what I mean. I don’t think you can expect a projection system to know when a team will or won’t play to its run differential. So now we’ve accounted for almost a quarter of the difference between the projected wins and the actual wins.

Yet, if you take away 13 from 55, you still have the Cubs in last, at -42 wins. So it’s clear there’s more to explain. I decided to take a look at team clutch performance, as measured by our in-house Clutch statistic. (You can find it on the leaderboards.) It’s true that Clutch isn’t a perfect stat, capturing everything. It’s true this could be partially captured already by under-performing the Pythagorean record. But I thought there could be something here, and it seems like there is.

You Aren't a FanGraphs Member
It looks like you aren't yet a FanGraphs Member (or aren't logged in). We aren't mad, just disappointed. We get it. You want to read this article. But before we let you get back to it, we'd like to point out a few of the good reasons why you should become a Member. 1. Ad Free viewing! We won't bug you with this ad, or any other. 2. Unlimited articles! Non-Members only get to read 10 free articles a month. Members never get cut off. 3. Dark mode and Classic mode! 4. Custom player page dashboards! Choose the player cards you want, in the order you want them. 5. One-click data exports! Export our projections and leaderboards for your personal projects. 6. Remove the photos on the home page! (Honestly, this doesn't sound so great to us, but some people wanted it, and we like to give our Members what they want.) 7. Even more Steamer projections! We have handedness, percentile, and context neutral projections available for Members only. 8. Get FanGraphs Walk-Off, a customized year end review! Find out exactly how you used FanGraphs this year, and how that compares to other Members. Don't be a victim of FOMO. 9. A weekly mailbag column, exclusively for Members. 10. Help support FanGraphs and our entire staff! Our Members provide us with critical resources to improve the site and deliver new features! We hope you'll consider a Membership today, for yourself or as a gift! And we realize this has been an awfully long sales pitch, so we've also removed all the other ads in this article. We didn't want to overdo it. In case you didn't read all that, here's the short version. Membership includes:
✓Ad-free browsing
✓Unlimited FanGraphs & RotoGraphs articles
✓Historical & platoon projections
✓Leaderboard & player-page stat heatmaps
✓One-click data exports
✓Customizable player-page dashboards
✓Dark & Classic site modes
✓Weekly Members-only mailbag
✓FanGraphs Walk-Off, your year in review
✓Optional homepage photo removal
✓Gift articles to non-Members
✓Full postseason coverage

For the 10 years, I combined both batter and pitcher Clutch score. The average, by design, is zero! One standard deviation is about 16 points. The Cubs are found at -27, ranking second-worst in all of baseball. The one team with a lower score is the Rockies, at -36, and it turns out they, also, have under-performed preseason projections over the decade. So at least by this rating, the Cubs have been the second-least-clutch team over a 10-year span, and it stands to reason that would have a significant effect on final record. It also stands to reason this can’t be predicted. I know the temptation is to think 10 years is enough to cancel out most of the noise, but Clutch seems to be random. If you know the key to harnessing it, I’m all ears, but I doubt you’re going to say anything, because if anyone actually understands how to be consistently clutch, that person works for a team. Maybe the Baltimore team.

It was pointed out in the comments that the Cubs have frequently been sellers, especially in the last few years. It seems like that could be a contributing factor — sellers lose good players halfway through the year, while team projections never count on trades taking place. Feels like a really obvious point. But here’s the interesting thing: over the decade, in the first halves, the Cubs have won 46% of their games. In the second halves, they’ve won just under 48% of their games. In the first halves alone, there’s a difference of 36 wins between projections and performance. So if trades are a factor, they seem like a really small one. The Cubs have mostly under-performed even before selling parts off.

I should note that, while the Cubs are at -55 wins overall, it of course hasn’t been even every year. They more or less matched projections in 2007 and 2014. They exceeded projections in 2008. Out of the seven seasons remaining, no disappointment has come close to the 2006 disappointment. In that year, the Cubs won 18 fewer games than they were projected to in March.

There’s a lot that goes into something like this, but I can easily identify some key factors. Derrek Lee missed most of the year with injuries, and the season before, he’d been worth seven wins. Mark Prior was limited to 44 mostly ineffective innings, because of shoulder problems. Kerry Wood didn’t even throw 20 innings, after seeming like a decent bet to bounce back. Injuries were devastating. The Cubs also under-performed their Pythagorean record by four wins, and in the summer they traded away Todd Walker, Greg Maddux, and Scott Williamson. They also dealt both Neifi Perez and Phil Nevin. Today I learned the 2006 Cubs had Phil Nevin.

So pretty much everything went wrong for that team. That’s the story of the Cubs’ biggest disappointment of the decade. There’s still a stretch from 2009 – 2013, where the Cubs undershot projections by 43 wins. I’ll point back to run differentials and clutch performance. There were also some injuries and surprising under-performances. In case anyone’s troubled by the pattern, the Cubs last season did what they were supposed to do. Which provides some hope this year’s team could be able to match its relatively lofty expectations. Again, probably the biggest point: while it’s been, overall, a rough decade for Chicago, there’s not great evidence that under-performing is predictive. Even if I haven’t completely explained the last 10 years, it doesn’t seem like it should mean anything for 2015. It just means Cubs fans would be extra appreciative of a competitive product that doesn’t fall apart.





Jeff made Lookout Landing a thing, but he does not still write there about the Mariners. He does write here, sometimes about the Mariners, but usually not.

40 Comments
Oldest
Newest Most Voted
Matthew
11 years ago

How many are those Angels wins from failing to predict Mike Trout.

Pennsy
11 years ago
Reply to  Jeff Sullivan

Why is it irrelevant just because it’s zero? Hypothetically without Trout they underperform their projections enough that their ‘lead’ over the first seven season of the sample doesn’t hold. I’m willing to believe Trout isn’t the largest factor over the entire time span but to say that he’s not the least bit responsible for the difference doesn’t seem right either.

glenstein
11 years ago
Reply to  Pennsy

I had to read this several times to understand it. But I think there’s an interesting point here, despite the kneejerky downvotes.

It sounds like you’re saying that, with the Angels at +47, they were “due” to get dragged back toward 0. So we should have expected some underperformance in the random noise to help drag them down.

And then suddenly Mike Trout came along, and “got in the way” (so to speak) of all the random noise that was supposed to drag the Angels down.

It’s an interesting idea. But I still don’t think it’s right. The weirdness is in getting to the +47 in the first place, not in the failure to of it to be canceled out by noise after the fact.

DD
11 years ago

Jeff, do you think you can tie it to individual players, like Zambrano, Prior, Wood, and Soriano? Seems like their production in particular was well below expectations after establishing a high ceiling.

Also, wouldn’t the Clutch factor and the pythagorean factor overlap in their effect?

Dave StudemanMember since 2017
11 years ago

Adding up clutch scores by team is a great idea–and it’s so somehow appropriate for the Cubbies to rank highly.

I assume there is no easy way to compare player-specific projections vs. performance for each team? that would be fascinating too.

Always ask for more, that’s my motto.

mario mendoza
11 years ago

Well annointing all their prospects as stars again this year isn’t going to help reverse the trend.

The Dude
11 years ago
Reply to  mario mendoza

Because that’s exactly what Fangraphs has done.

McP
11 years ago
Reply to  mario mendoza

Very true. I think the Cubs disappoint me again this year and fail to live up to expectations. I don’t know the history of prospects going bust/becoming stars, but it seems like the Cubs won’t hit on all of these young players coming up. They are very promising, but expecting all of them to become even everyday contributors seems like asking too much.

dl80
11 years ago

Is it possible that many players underperformed in the first half and then were sold off before they could regress back toward the mean? Seems like it wouldn’t matter that much, but just wondering.

meathead
11 years ago

it’s almost like they’re…cursed

Statistics don't lie
11 years ago

Come on guys, anybody can see it – IT’S THE CURSE!!!

Dave StudemanMember since 2017
11 years ago

Jeff, can I ask a dumb question? I can’t find the answer anywhere. Are Fangraphs’ team projections based on a simulator? Or are they a sum of each player’s projected WAR and playing time? Or something else?

Dave StudemanMember since 2017
11 years ago
Reply to  Jeff Sullivan

Thanks, Jeff. Wish me luck getting David to respond to me. Also, I’m not a robot.

Jonathan Reimer
11 years ago

While I’ve always struggled to understand the fascination with Mike Scoscia, this graphic has to give him overdue credit. He’s managed the Angels through the entire graphed period and can take some credit for coaching his teams to [a lot] more wins than talent levels have predicted.

Justin
11 years ago

Why? Is this like a manager of the gaps argument? I don’t managers should automatically get credit for any unexplained over-performance.

Justin
11 years ago
Reply to  Justin

*I don’t think managers…

Jeff
11 years ago

Joe Maddon will fix this.

Ken
11 years ago

“Today I learned the 2006 Cubs had Phil Nevin.”

I had almost forgotten that. Thanks.

MGL
11 years ago

Comparing pre-season projections to actual wins is a worthless exercise because you don’t know the playing times of the individual players and the personnel in general. That is the only thing that counts, projection-wise. Pre-season, you guess at personnel and playing times because that is all you can do.

If you ever want to analyze why a team under or over-performs projections (and plenty will do that by luck alone), you MUST start with individual player projections prorated for actual playing time. If you don’t, and you find teams who are way ahead or way behind of their team pre-season projections, like the Cubs, you are likely simply finding teams whose pre-season personnel and playing time estimates were wrong.

MGL
11 years ago

Before you start giving managers or magic fairy dust credit for over-performing or under-performing, you do two things:

One, as I said, you use player projections and actual playing time. Those player projections MUST be comprehensive – include base running, defense, etc.

Then, you compare the distribution of results to what you would expect given that all deviations were random. For example, the graph in this article looks a lot like a normal one that you would expect by chance.

If you find that the actual distribution is similar to what is expected by chance, then you assume that all deviations, large or small, are by chance alone.

If there is some difference between that expected by chance and that observed, then you can easily figure out how much to regress the observed deviations in order to estimate some kind of “skill,” be it manager or otherwise. But there is always going to be a regression and likely a large one. So, for example, if there IS a difference between the observed and the expected (by chance alone) deviations from pythag or actual win% based on player projections (again, assuming that the projections are comprehensive and robust), it is likely that you would regress the Angels 4.5 wins per season quite a bit (maybe 50-90%) if you want to start assigning credit to someone or some thing.

(Again, my guess is that you will find that the Angels projected win% is much greater if you use player projections prorated to actual playing time, and vice versa for the Cubs.)

Jared CrossMember since 2016
11 years ago

The spread in totals seems somewhat surprisingly small. Neil Paine had Vegas with a RMSE of 9.1 since 2005 and if the year’s errors were independent we’d have a RMSE of 9.1*sqrt(10) = 28.8 over 10 years and we’re only seeing a SD of 21 in those totals above. Not that I endorse this way of thinking but that SD is smaller than what we’d expect by a (just barely) statistically significant extent. Probably just a funny result but it looks like over these 10 years Vegas’s misses on teams have an inverse association.

Brett W
11 years ago

Mr. Sullivan, you are probably my favorite baseball blogger in the game as of 2015, but since you do not read Cubs-specific blogs such as the esteemable The Cub Reporter, you clearly were not aware that this phenomenom was identified and classified years ago.

Hence:
http://www.urbandictionary.com/define.php?term=Cubbery

Root symptoms of Cubbery: Gross macro-management, gross management, gross mismanagement, foul winds, Michael Barrett

RMD
11 years ago

The cubs and mariners are las due their crazy schedules. The Cubs are won out by all those days games and the Mariners have to travel more than any other team. It makes sense they tend to underachieve.

florida ron
11 years ago

It appears Theo Epstein and Hoyer had little faith in the underachievers. From the 2012 roster of just 3 seasons ago, you have only Castro, Rizzo(played in just 87 games) and Castillo(about to be traded it seems), and amazingly zero pitchers remaining from 2012. From the 2011 squad, only good old Starlin remains. A complete and utter teardown and rebuild.

shthar
11 years ago

Cubs Fever! Catch it and DIE!

roger
11 years ago

How much is directly attributable to blown saves? Seems like the Cubs have always trotted out a Veras or Camp to implode after a very quality start? Also, how much is merely divisional reciprocal? Are Angels beneficiaries of playing the Mariners and the Cardinals/Reds/Brewers beneficiaries of playing the Cubs? Any way to adjust for unbalanced schedule?

1908Member since 2016
11 years ago

And let’s not forget Ronny Cedeno’s epic 2006 campaign: .245/.271/.339 sprawled across 572 ghastly plate appearances. His 54 OPS+ tied for fifth worst in the 2000s. Any projection system that could have predicted this would probably have self-destructed in existential despair.

Timbooya
11 years ago

“If you know the key to harnessing it, I’m all ears, but I doubt you’re going to say anything, because if anyone actually understands how to be consistently clutch, that person works for a team. Maybe the Baltimore team.”

Personally, I think of the Giants when I think clutch. Bumgarner’s performance, Affeldt during the playoffs, Pablo tying up the game with 2 outs and 2 strikes in the ninth in Washington in the NLDS… Not to mention the other two WS wins and those clutch moments.

Pure
11 years ago

What that graph tells me is projections do not work. I am willing to bet I could get a bunch of chimps to make projections and they would be just as accurate as any current system.

TheGrandslamwichMember since 2020
11 years ago
Reply to  Pure

I’ll take that bet. And I’m willing to put 5 grand on it.

Dan
11 years ago
Reply to  Pure

as long as you don’t name any of the chimps Marcel you should be fine

florida ron
11 years ago

Yeah, what it really says is you guys are pretty bad at making projections.

Jared CrossMember since 2016
11 years ago
Reply to  florida ron

Huh?

This record is remarkable good. An SD of 21 over 10 years! This corresponds to an SD of 21/sqrt(10) = 6.63 in any one year if they’re independent. If you knew the true talent of ever team ever year exactly you’d have an SD of 6.4. Don’t get me wrong, I think there’s some luck involved in this 10 year record and maybe things aren’t completely independent (although they looked very nearly so in an earlier post) but I don’t understand how you can look at those numbers and see bad forecasting when, if anything, the record is surprisingly good.

What kind of SD were you expecting? In other words, what range would make you think that the projections were good as opposed to bad?

David Bowser
11 years ago
Reply to  florida ron

And this is where everyone gangs up to remind you to read the original article. The projection data were based on SEVERAL different systems, so “you guys” refers to a whole bunch of people, not just FG. As has already been pointed out, these projections are quite accurate, and therefore all the discussion has been around the outliers (6 teams including Cubs).

hookstrapped
11 years ago

Maybe they should be called Preseason Power Rankings instead of projections.