Putting the Nationals Disappointment In Context
Other teams are clinching divisions, and Nationals players are fighting one another. Not that players on good teams haven’t fought, and not that the Nationals’ incident was in some way unique, but this isn’t how it was supposed to go. A few days ago, the Nationals were mathematically eliminated from winning the division, and in reality it feels like it’s been even longer. They haven’t been within three games of the Mets since August 11. They’re currently behind by 9.5, and only the AL Central has a bigger gap between first and second place — by a half-game. It’s not an exaggeration to say the Nationals are feeling pretty embarrassed.
This is all review to you, but it’s not like the Nationals were simply the favorites. They didn’t come in with slightly better odds than any of their rivals. After the Max Scherzer acquisition, Bryce Harper talked about rings, and the only issue there was Harper was expressing what the rest of us thought. It looked like the Nationals would win the East in a landslide. Our projections figured as much. The PECOTA projections figured as much. The Clay Davenport projections figured as much. The whole entire media figured as much. And it wasn’t just about looking forward. Between 2012 – 2014, the Nationals won more games than any other team in baseball. They had 280 wins, and their closest division rival had 269. Those were the Braves, who were rebuilding. The Nationals looked good, and they didn’t have a serious threat.
It’s not like I need to go over everything that’s happened. We’ve all lived it, and we know how the Mets have surprised, and how the Nationals have under-achieved. Those stories are old stories, but now that the Mets have actually clinched the division, I wanted to re-visit something. I wrote in March about the Nationals’ projected division advantage. Time to go back to that. How significant is what we’ve seen happen?
There’s no way to pull numbers out of qualitative expectations. Team projections are the best estimates we have, and right before this season started, we had the Nationals projected to finish 13 games clear of the Mets, and 14 games clear of the Marlins. You’ll recognize that as a huge gap. And it’s indeed a very huge gap. Some time ago, I assembled team projections going back to 2005, and though methods have changed over the years, ultimately they’ve all been measuring the same stuff. So I put them all into the same spreadsheet. What we have now are 11 years of team projections. That’s 11 years with six divisions in each, so, that’s 66 projected division leads. In the plot below, you’ll see the projected differences between first and second place. This isn’t really different from the image you saw in March, but now, of course, we know a little something about what happened in the NL East.
66 divisions. 53 projected to be decided by no more than five games. 64 projected to be decided by no more than 10 games. So, two are left. In 2005, the Cardinals were projected to win the NL Central by 17 games, and they eventually won by 11 games. And in 2015, the Nationals were projected to win the NL East by 13 games, and they’re going to lose, by a lot of games. Not even by a small margin! They’re going to lose to the Mets by a difference in the neighborhood of how badly the Mets were supposed to lose to the Nationals. Distilled: the Nationals had the biggest projected division advantage in a decade, and they didn’t win the division. They did worse than they were supposed to, and the Mets had a lot go right, despite how horrible things looked in July. (Even in July, the Nationals were in first.)
And as you know, the Nationals aren’t just settling for something other than a trip to the NLDS. No, they don’t get the wild card, either. On the eve of the regular season, we gave the Nationals 87% division odds, and 95% playoff odds. That leaves some room for a playoff miss, but it doesn’t mean it’s not extraordinary. This always had a chance of happening, but it had less of a chance than it almost ever does.
There is a way to make this a little less extraordinary. Since, again, 2005, we have 330 individual team projections. That’s 11, for all 30 teams. The Mets are on track to beat their projection by about 11 games. Each year, an average of almost four teams does that. And the Nationals are on track to miss their projection by about 12 games. Each year, an average of about three teams does that. This year alone, the Nationals are on track to miss by 12 games, but so are the Mariners. The A’s are on track to miss by 15. The Brewers and Tigers are on track to miss by 10. Only two of the projected division winners are going to win their divisions.
So if you want to think of the Nationals as the biggest bust ever, there’s competition. They’ve blown what seemed like a huge division advantage, but if the Mets played to their expectations, the Nationals would still be in this in the final week. Just compared to their projected win total, the Nationals aren’t a bigger bust than this year’s Mariners or A’s. And going beyond 2015, the 2012 Red Sox missed their projection by 22 wins. The 2005 Dodgers missed theirs by 21. The 2009 Indians missed theirs by 21. The 2010 Mariners, 20; the 2014 Red Sox, 18; the 2008 Tigers, 17. There’s a long list of disappointments. They didn’t all project for first place, but they all won less than they should’ve.
Just in the division sense, the Nationals have been an enormous bust. On an individual-team sense, they’ve been a lesser degree of bust. But that doesn’t mean they haven’t been a bust, and that doesn’t excuse them for their performance. Recency bias makes them look worse than they really are, historically speaking, but they don’t look good, and they appear to have real problems. Problems not just with imminent free agencies, but problems within the clubhouse, problems that might have to do with the leadership. Previously, the Nationals would underachieve in October. This year, they got it out of their system early. Their mission is to figure out why this went how it did. There’s nothing they could’ve done about the rise of the Mets, but they could’ve and should’ve stopped their own sinking. Instead, despite one of the biggest individual breakthrough seasons ever, the Nationals get to think on what the hell just happened.
Jeff made Lookout Landing a thing, but he does not still write there about the Mariners. He does write here, sometimes about the Mariners, but usually not.
I assumed that this was just going to be a picture of Paplebon choking Harper in the dugout.
Harper is not the MVP. Arrieta has led his team to the playoffs
Arrieta might not even win the Cy Young.
You’re new here, right? Arrieta is in the playoffs because he has better teammates, not because he’s more valuable than Harper. If all it takes is making the playoffs, I assume you also think that Castro is a better choice than Harper!?!?
Or me!
Considering he’s Bernie Sanders, he probably DOES prefer Castro 🙂
He’s not Bernie Sanders. He’s bernie Sanders. Can’t you read?
You’re a baron, I’m a colonel, we’re all busy people here…
Ruining Sanders’ name. Should be ashamed.
Cubs probably make the playoffs with or without Arrieta. In fact, pretty much every team would be where they are in relation to making/missing the playoffs in the NL if you subtract any one player from their roster. Guess the NL doesn’t have an MVP this year.
Your feeding a troll! bernie Sanders is a troll!
He sucks, he could only lead his team to third in his division. Hell, Mike Stanton did this and he didn’t even play the last 2 months of the season. Let’s give the award to real men who were able to lead their teams to division championships. Give it to Prince Fielder!
Question of the day – if the Nats traded Papelbon and Williams for absolutely nothing, what would they have to throw in to make the deal fair?
Toss in Harper. You wouldn’t want the adopting team to break up the siblings, right?
A time machine to make Williams 20 years younger.
Actually, they should hang onto both of them until the deadline for upcoming free agents to accept qualifying offers. It gives them the chance to offer Span and Fister QO’s with a low probability that they’d actually except (and yes I know Span would be coming off hip surgery, but still….).
Then jettison them both ASAP.
Touki Toussaint
Dan Snyder
I thought this was a safe place free from the mention of such atrocities. Using Snyder’s name on a baseball site should come with a trigger warning!
I didn’t realize I was on a gutless Little League team!
The Phillies are about to lose 100 games, and they’re not even the most embarrassing team in the division.
And to think….I would have paid them TWICE Papelbon’s salary just to get rid of him.
Dumbasses.
– collapse of Ian Desmond, Doug Fister, Jayson Werth; second-half collapse of Max Scherzer
– injuries to basically everyone except Harper
– bullpen issues (inc. injury to Craig Stammen)
– Matt Williams’ tactical, strategic, and personality-management ineptitude
Did you see the report that Matt Williams decides what relievers he will use, in which order and in which innings, before the game even begins? I think that, alongside the Papelbon incident, sums up the team’s woes.
Any truth to the story that Rizzo got Papelbon because Williams insisted he needed a closer he could trust instead of Storen? I seem to recall that was also the case for getting Soriano too.
I don’t know about that one, but in my corner of the Nationals internet (WNFF.net) there was a lot of unrest that the target was Papelbon rather than 1-2 of the Padres setup guys.
Injuries are a poor excuse for this Nats team. Everyone of the Nats projected starters or starting pitchers who missed time this year had a significant history of injuries.
Of the regulars, Werth, Zimmerman, Span, and Ramos have all missed at least 30 games during two or more of the four years preceding this year. Rendon fell to them in the draft due to injury concerns. And Strasburg’s injury history is well documented.
Whoops, didnt mean to reply to this comment, meant for the general thread
Papelbon you are a disease to this team.. I am MVP and your worthless. I wish I was a met..
Can you link to that report?
I don’t think you can blame injuries as the reason they lost the division to the Mets.
By all accounts, the Mets had just as bad injury luck. David Wright missed 120 games. Travis d’Arnaud missed 80 games. Daniel Murphy was out for all of June. Zack Wheeler has missed the entire season due to TJ surgery. Jennry Mejia, the presumptive closer, has been out pretty much all year due to two steroid suspensions. Josh Edgin, the primary LOOGY, had TJ surgery. Then Jerry Blevins, the LOOGY they traded for after Edgin got hurt, has been out since April with a broken arm.
If anything, the Mets have had worse injury luck than the Nats.
With injuries being a wash between the two teams, the real reason the Nats collapsed was underperformance and inflated expectations.
People banked on Jordan Zimmerman being an ace when his career peripherals showed a mid-rotation starter. Fister and Roark were expected to be mid-rotation starters when their 2014 peripherals were those of fringe major leaguers.
Rendon was coming off a career year that everyone expected him to replicate (he didn’t). People simply assumed that Werth would never show his age. Nobody noticed that Desmond had a career high strikeout rate in 2014, a major warning sign.
People didn’t realize that their bullpen was massively understaffed with the Clippard/Escobar trade, and that Escobars defense had declined dramatically in 2014 (a sign of future troubles at 3B).
The Nats had their flaws. But people were so enamored with a Scherzer/Strasburg/Zimmerman/Gonzalez/Fister rotation that they looked past these flaws (and past the less than stellar numbers of the back of that rotation).
While you make several good points, I think you’re being too harsh on the “inflated expectations”.
Who was banking on Zimmermann being an ace when the rotation already had Scherzer and Strasburg? He didn’t need to be an ace, if he could be as good as he was in previous seasons that would be amazing. As it was, he had his worst season since being a rookie. Should that have been expected?
Roark had a FIP- of 95, that’s a fringe major leaguer? He was the 6th man anyway.
While saying Rendon was coming off a career year is technically correct, it kinda loses meaning when that career consists of a grand total of two seasons. This was a guy who was originally projected as the number one draft pick mostly based on his elite hitting skills. Regression was expected, but all the way down to barely above replacement level?
I think the criticism of constructing a roster with a ton of injury concerns is more valid. Most of the underperformances shouldn’t have been expected, at least to the degree that they were.
Except they missed more starter DL days than anybody in baseball, so their injuries were objectively worse. Consider: Nats had their opening lineup 1 day this year. Mets had opening lineup+ hug additions for a decent chunk of the year. Rendon=3 DL stints. Werth=2 DL stints. Ryan Z=2 Dl stints. Strasburg= 3 DL stints. Span=3 DL stints. That’s just looney tunes. Sorry. I do think Rizzo deserves blame for not supplementing depth and counting on Ryan Z, Span to stay healthy once they returned.
Then again, the Cardinals have lost their projected #1 starting pitcher, primary setup man, starting LF, and starting 1B for most of the season, another starter had 2 different DL stints, and both the starting CF and his replacement have also missed significant time, yet they’ve won 100 games with 3 to play. Go figure.
The Matt Williams bullpen thing can’t possibly be true, can it?
Shouldn’t this be more damning of the projection system then the team itself? I mean we knew at the outset that Rendon and Werth had issues to start the season, and the write-up for Zimmerman on this site even calls out his injury risk, yet each was projected for 500+ PAs. And on top of the projected availability, the projections also assume a performance level that doesn’t reflect the relative health of players starting the season injured.
Go look at the numbers, it wasn’t the pitching or defense which let WAS down, it was their hitting, and a lot of that underperformance vs. projections is the fault of the projections.
That’s right. It’s the projections fault, whatever that is.
Ok, that is funny.
But seriously, it was clear in March that projecting the Nats to run away with the division was faulty….so why is it a story now that a faulty projection was so far off base? I mean it was off-base to start the season.
I see your original point. I don’t think projection systems can ding a player’s numbers for the aftermath of any injuries, and they probably are too aggressive in assigning expected playing time for those guys as well.
But IIRC Jeff had an earlier article that showed how even a lot of the bench guys way underperformed their projections (on a prorated basis). So, those optimistic projections only exacerbated the impact of all the injuries to the regulars.
A perfect storm.
Rendon’s injury wasn’t supposed to be that bad. Werth’s injury was the only one most were really worried about and Taylor was seen as a capable back up. Zimmerman was an injury risk, but he was the only one I remember hearing about coming in. I thought the projections were fair even if the offense proved mediocre. They had 6 very good starters. They should have been a 90 win team.
It’s a projection system, not a prediction system… huge difference.
Projection, not prediction? That’s doubletalk, is what that is. It’s the same horsefeathers Szymborski serves up when his projections are badly off target.
Is this a semantic equivocation or a distinction without a difference?
You should look up the difference between maximum likelihood and unbiased estimation methods.
I really hope we stop wasting so much e-ink on projections next year, because the overarching narrative of FG this year has been mild annoyance that reality didn’t match those projections.
It’s certainly true that health and injury factors aren’t adequately accounted for in the projections. It’s probably also true, though, that they’re in large part a matter of blind luck.
It’s human editors at FanGraphs, not a projection system, that decides on the playing time estimates. Projection systems usually do give estimates of playing time, but those are not used at least here at FanGraphs because the idea is that humans can predict playing time better.
Totally agree. Projections have errors. Intellectually everyone knows that. But the reality is that if you look at the pre-season projections here, every single author also projected the Nats as winners. Those who have the most knowledge of both statistics, probabilities, and those projections fell into the same trap as people who are completely innumerate. If a number is presented, it must be as certain as 2+2=4. How can there possibly be an error (or even a range?)- there’s a number there and the math is correct right?
Is this conversation serious?
“Errors” aside, projections have uncertainties. Every single projection ever created is based on assumptions and therefore has uncertainty. The results of the projections are generally simplified to show the most probable result, but that doesn’t mean there isn’t some probability that the actual result will be higher or lower than the projection. The projection always projected that the Nationals could have an 83-win season. It just didn’t project that with a high probability.
Can’t believe this is coming up on Fangraphs.
Yeah, and this is essentially why I have criticized the “Blow it Up” strategy that a lot of baseball analysts and FG commentators seem to favor nowadays. As Jeff shows, there is a lot of variability in the projections– what was the joint probability at the season’s outset that the Mets would overperform and the Nats would underperform? I haven’t done the messy research on that, but I’d guess it was a nontrivial chance. In the Mets’ case, it was more a matter of coming on faster than expected rather than random over performance, but I suspect other teams that were criticized pre-season for trying to be competitive, notably the Reds and the Brewers, also had nontrivial chances at making playoff runs.
Of course it should come up on Fangraphs. You can blabber on about uncertainties – but the reality is the authors on this site were UNANIMOUS in selecting the Nats to be division winners in their pre-season projections. Nats were not 100% certain to be the division winners – as time proved and as they all knew as well if ‘87% chance of winning division’ was info they had available.
Every single one (I think there were about 20 or so) of those authors chose the favorite to win. And in so doing absolutely created the false impression of certainty about those projections.
Just because there’s an 87% chance they win the division, that does *not* mean that 87% of pollsters should choose them. What it means is that 100% should choose them, but they would be wrong 13% of the time. The *only* reason to have chosen the Mets is to be a contrarian.
Think of a die with 6 sides. #1 is on five sides, and #2 is on the sixth side. If I tell people I’ll give them $10 if they guess which number will come up, every single person should say #1 (unless, as mentioned above, they’re trying to be contrarian even though they’ll most likely lose). That doesn’t mean we’re 100% certain that #1 will come up. We’re not creating the false impression of certainty that #1 will come up. We’re just playing the overwhelming odds in our favor.
Actually that’s wrong. the Nationals led the NL in runs scored this year. Their starting pitching was incredibly mediocre and the bullpen was a dumpster fire.
“We’re the team to beat, everybody knows that. And we’re going to get hot. Everybody knows that too.”
Jonathan Papelbon knows that, too.
Bernie Samedi dump Papelbon I am the man MVP.. Or please release me to the NY Mets..
I am just really looking forward to the projections for the next six days.
I don’t think team chemistry always matters, but in some cases it does. And in the Nationals 2015 season, it did — as epitomized by the Papelbon trade.
The Papelbon trade seems to point out a blind spot on the part of Rizzo concerning team chemistry and intangibles.
Speaking as a Mets fan, I hope Rizzo is not replaced anytime soon.
Nah man, as a Mets fan I hope he gets canned. Hiring and sticking with a shitty manager is a mistake he can learn from. Acquiring a guy who ruins morale or chemistry is a mistake he can learn from. He’s pulled off some great trades and has a good track record of identifying and developing talent. Those are traits that can’t be easily taught.
How hard is it to identify and develop talent when you have the #1 pick in back to back years? Plus the Lerners’ big money to support your plans? Take away Harper and Strasburg and it’s a much different team.
I would love him to stay, along with Papelbon, who is a disruptive force. Unfortunately, it looks like Papelbon is on his way out this winter. The Nats have already removed his jersey from their team store. That was fast!
Sorry but Strassburg has not been anywhere close to THE reason for this teams success in recent years. He has been good, but no where near 1st pick overall good. Last season 2014 he was 4th among Nationals starting pitchers with 30 or more starts in ERA. 2013 he won all of 8 games in 30 starts. in 2012 he was 3rd in ERA for Nationals pitchers with 25 or more starts and he only had 28 starts. That means he has basically been a very valuable contributor, but has nowhere near carried the pitching staff ever!
Harper in 2015 I will give you.
If you look at cumulated WAR since Strasburg first pitched in the majors 2010. Strassburg is 20th amoung pitchers and 58th amoung all players since he entered the league. In other words, very good, but probably not what you hope for from a first overall pick.
At least with his degree of hype anyways!
Here are the #1 picks since the strike and their career rWAR through their first six professional seasons:
18.6 2007 David Price
13.9 2001 Joe Mauer
13.5 2009 Stephen Strasburg
11.8 2005 Justin Upton
08.2 1998 Pat Burrell
07.7 1996 Kris Benson
02.3 2000 Adrian Gonzalez
01.5 2003 Delmon Young
00.5 2006 Luke Hochevar
00.2 1997 Matt Anderson
00.1 2008 Tim Beckham
00.0 2004 Matt Bush
00.0 1999 Josh Hamilton
-0.3 2002 Bryan Bullington
You’re right– his performance does look out of place among #1 picks
Moves Rizzo has made that had nothing to do with Strasburg and Harper:
–Dealing Souza for Trea Turner and Joe Ross (robbery)
–tradingfor Wilson Ramos (good move as of now, altho Ramos has been bad recently)
–Dealing for Gio Gonzalez with 3 nobodys and being rewarded quality production and one Cy Young candidate year (until this year)
–Drafting Rendon (might end up being best hitter from that draft class)
–Drafting Jordan Zimmermann (huge)
–Drafting Giolito (going to be a stud).
–Dealing Alex Meyer for Span (huge plus so far)
–Trading nobodys for Doug Fister (Cy Young candidate last year before imploding this year)
–Sticking with Ian Desmond when everybody said to drop him.
–signing LaRoche (excellent production, all things considered)
–signing Werth (actually, if you look at his WAR, pretty close to justifying contract)
–Scherzer (TBD)
–Soriano, Papelbon (1st was eh, second was disaster)
–signing Edwin Jackson for a year in 2012 (best year of his career basically)
–dealing a nobody for Asdrubal Cabrera 2014 deadline (played very well)
So….on balance I respectfully disagree.
So projections really loved the 2005 Cardinals. I would love to see more analysis of that specific projection.
They were kinda good: Pujols, Rolen, Edmonds, Walker, Sanders and of course Eckstein and Taguchi. How Taguchi was he? So Taguchi.
http://www.fangraphs.com/leaders.aspx?pos=all&stats=bat&lg=all&qual=200&type=1&season=2005&month=0&season1=2005&ind=0&team=28&rost=0&age=0&filter=&players=0
+1 because my little brother and I used to say “How Taguchi? So Taguchi!” all the time and it made me nostalgic.
Imagine trying to explain The Muni Kawasaki Epoch to people in 10 years?
“You see there was this middle infielder. And he was a total weirdo goofball spazz. And it was really funny to see him be such a spazz, considering he was a Major League ballplayer, but his spazziness was somehow endearing to the fans anyway, partly because it was so funny. Like, he once tried to pig pile on Chone Figgins, but instead of really pig piling he gingerly laid down on top of him. It was funny!”
Not that hard.
Did you really just describe Muni without mentioning that he is Japanese? You know, he is not ashamed of being Japanese. You won’t hurt his feelings. He is proud of the fact. Or are you afraid of running afoul of people who are offended for other people on the Internet (even if the people for whom the vicarious offending is felt wouldn’t themselves be offended)?
And that was a terrible description as well.
They also had a dominating starting rotation of Carpenter, Mulder, Morris, Suppan, and Marquis.
While the Haren for Mulder trade eventually became a disaster (perhaps the worst trade Jocketty ever made for the Cards, A’s, or Reds), and Haren himself had a pretty good 2005, Mulder did have that one great year for the Cardinals before injuries tanked his career.
Imagine if Yuniel hadn’t had a career year and Ryan Zimmerman hadn’t caught fire for a bit after the break?
Well,you can imagine anything. Imagine if other players played well, or if those particular players played less well. The team record would likely be different. I think that’s the point of this article.
Actually, this should have happened more frequently. A 5% chance happens one time in 20 tries. Since we went back 10 years, given the number of teams, this should have happened more than once, and it probably has.
I’m sure in some year team X had a 95% chance to have a certain outcome and failed to make that outcome. The White Sox weren’t division favorites, but I bet their odds of losing this many games was not much better than 5%. Same with Padres.
I was thinking along the same lines, but in another way. Based on
…we’d expect this wild of a swing in (4/30)*(3/30) = .013 = 1% of the time, or about once a century.
If, in any given year,
So the 2015 NATS are the team of the century!!!!!
The Mets outperformance and Nats underperformance are not separate unlinked events. They play in the same division. One times loss is usually a division rivals gain
True, and also, I suspect that there are characteristics of over-performing teams and under-performing teams that alter those probabilities. I’d guess that young teams with lots of talent in the higher minors are over-represented in the sample of over-performers, and old teams with aging veterans are under-represented. It probably also depends on factors like hitting-pitching balance, stars-scrubs balance, etc., as well as the variabilility of key player projections.
The other thing to consider is the Nats run differential is similar to the Cubs and Dodgers and Mets. There’s a lot of run sequencing luck in here (and horrible leveraging of close situations by the smartest manager ever).
Fangraphs said the Red Sox and Nationals would win it all. They bombearded us with deaily articles in the offseason about their dominance. If Williams has to own the failure of the Nats, I want someone at Fangraphs to own the failure of their projections.
All I take away from this is bombearded
Not just bombeards, but Deaily Bombeards!
I could use a Bombeard 🙁
Deaily Bombearded, we are gathered today in the presence of God and these witnesses…
¡Jajajajajajajajaja!!!!!!!!!
That was my nickname with the Tigers last year!
If you do too many bearbonggs you get bombearded…
Me too, I am beardless. WHERE’S MY BEARD!!!
Fangraphs says Red Juice and citizens will win it all. We bombearded with deai text in the offseason about his dominance. If Williams must have to fail, I want someone Fangraphs failure of their predictions.
deaily articles in the offseason about their dominance.
This definitely happened.
Red Sox may very well have won if not for Ortiz and Porcello slow starts and injuries to Buchholz, Hanley and Pedroia, not to mention the complete collapse of Napoli against RHPing.
I think their play in the last 2 months has partly vindicated Ben and FG. Lot of moving parts just did not come together till late.
Nats was just bad managing and a couple of down years . With that weak division they play in (Harper hit half his HR agains the Braves, Phillies and Marlins), they should have won 95 easy
Couple of down years by some players
Only Love, No Glove
Damn it!
Not sure that an injury to Hanley hurt the Red Sox. If anything it made them better because they could play someone who wasn’t the worst player in the league
Tough to say, he only started playing like the worst player in the league after he hurt his shoulder. Before that, he could still hit at least.
He hit really, really well in the beginning of the season. Then he ran into the wall.
As I said, Nats are a disappointment, but if you believe in run differential as more accurate indicator of talent performance, their run differential is similar to dodgers, mets, and cubs.
Suggestion: the projection systems are fine, but the running of the 10,000 simulations is flawed — because the simulations _take it as a given that the projections are correct_. In every single simulation, the question is “What will happen to this .548 team?” “What will happen to this .481 team?” … leaving no room for “Something happens to make this team better or worse”. The cool thing is? That’s easily fixable!
Rough-draft suggestion: in every simulation, before the games are run, the computer rolls every team a D6. The faces represent “add .030 to estimated team strength”, “add .020”, “add .010”, “no change”, “subtract .010”, “subtract .050”. This represents the team being healthier or less healthy than expected, prospects developing better or worse … and my top-of-the-head guess that maybe five teams a year run into substantially heavy injury problems. This season, for the Nationals, would then be a season where their actual collective ability came out .050 worse than expected — and even at that, they got a few games unlucky.
Could better numbers be picked than mine? Probably! But we have a projection system that’s overconfident, because it takes team ability as known and fixed. It would work better if it stopped doing that, and varied teams’ true strengths as well as their outcomes.
Well….that’s the thing about projections, they’re supposed to account for uncertainty. The output is just what the projection systems spit out as a number most representative of all possible outcomes. But those possible outcomes include the times things go wrong. You don’t re-weight the projections to account for that, that’s what projections are doing–accounting for uncertainty.
The problem is when people look at a projection and treat it as a tangible number that’s meant to reference an exact outcome. It’s not. It’s a representation of probability and uncertainty. The 5% of the the times the projection included the Nats missing the playoffs is supposed to account for the times they don’t wind up a .548 team (and/or another team does substantially better than projected). Whether the system does this well is a different question entirely, but the theoretical basis is sound.
With respect, you’re wrong, because the projections account for *one* kind of uncertainty, when in real life projections have *two* kinds of uncertainty. As I say: a lot of different things can happen to (say) a .590-quality Nations team over a season, and 10,000 simulations will capture most of them. That’s uncertainty level one. But they only capture results based on the assumed .590 quality!
They don’t account for what happens if those Nationals get seriously injured and are only (say) a true-quality .540 team — which seems to have happened, with their won/loss record lagging behind even that revised true quality. Or, for that matter, what happens if they get lucky with injuries, have Bryce Harper emerge into a superstar, and become a true-quality .620 team, which also could have happened — the Nationals were actually more likely to go 115-47 or something ridiculous like that than the projections suspected. And they were more likely to have their season go pear-shaped. Because .590 was only ever a guess at true quality.
Actually, the projections exactly do account for that kind of uncertainty.
The projections are based on past outcomes, and all past outcomes involved the types of variation in true talent you describe. Bryce Harper’s projection is based on past outcomes, which include the chances he truly breaks out and becomes a superstar, increasing his true talent. The problem is that, from a statistical perspective, it’s pretty hard to distinguish a true talent change from random variation. But, the projection is created to weight the better breakout indicators more, and also to include a true talent change in its range of possible outcomes, albeit as one of the least likely outcomes.
You are also, with respect, wrong.
The simulations do *not* include trial runs where Bryce Harper turns into a superstar, or is injured; they include only a projected 50th-percentile summary of his possible results. They do *not* include trial runs where Anthony Rendon and Jayson Werth are useless and Stephen Strasburg and Doug Fister are far off their game; they include single weighted projections that take into account the off-chance that they might happen. All of the variety in the 10,000 simulations is random chance: that sheerly by luck, a 95-67 team might win 105 or 85 games. Which is a start.
The “random noise” Roger accuses me of below is the random noise of real life. In reality, the Nationals weren’t a 95-67 quality team to begin with, becuase they were missing half their key players. The second layer of uncertainty I propose is that you should also, in considering the Nationals, be considering the chance that they’re a 100-62 team that might easily win 110 or 90; and that they might really be a 87-75 team that might easily win 77 or 97.
Same with the Red Sox: their confident projections allowed for random variation in Hanley Ramirez or Pablo Sandoval, but not for actual changes in their ability — which seem, after the fact, likely to have occured. Or the Royals: it seems likely that Mike Moustakas and Kendrys Morales and Lorenzo Cain are better than the projections assumed to begin with (although probably not as good as they look this year), so that instead of being an 81-81 team getting absurdly lucky, they’re more of an 86-76 team getting conventionally lucky.
Fangraphs’s projection system is too easy to surprise: we’ve seen that. It’s not because it’s wrong about Most Likely Outcomes; it’s that it only allows random variation around a known team quality, and it should allow a random variation about a varying team quality. Reality is noisy.
With respect as well, I’m almost certain you’re wrong about this. These are identical forms of uncertainty. Every simulation absolutely does NOT include Bryce Harper’s median outcome, for example.
By “Identical forms of uncertainty,” that is, I mean they are types of uncertainty theoretically accounted for. You can argue the projections don’t account for them effectively, but the theory is designed to account for the possibility that Bryce Harper has 200 PAs and 600 PAs, Strasburg has a 2.00 FIP and a 4.00 FIP, Werth hits an age decline and Werth doesn’t hit an age decline, etc. That is precisely what the uncertainty is representing. That’s what forecasting is. It accounts for hypothetical explanations for outcome variance.
Ok, just realized: I’m right, but there’s a reason from my experience why this isn’t obvious to most people. I’ll make one last try to explain, telling you about me, baseball Monte Carlo simulations, and how for a few weeks I kept myself entertained on three hours’ sleep with a baby on my shoulder.
Using a method Bill James invented in the 1990 Baseball Abstract, I would generate team strengths for entire leagues full of teams. I made up little rules so that the year-to-year progress of team strengths would be unpredictable but believable. Then I would rapid-simulate 162-games-per-team seasons. I kept records of the actual team strength and the resulting won-loss, I held playoffs, hey, you try to do better in that exhausted state.
So maybe a team, over three years, has its generated strength fall from .560 to .511 to .474. However, as happens now and then, fate is perverse; they get extremely unlucky the first year, and fairly lucky the third year, so their records actually *improve* from 79 to 80 to 85 wins. What happened? Did their best players get injured the first year? Did a couple of rookies overachieve the third year?
No. THE TEAM DIDN’T HAVE ANY PLAYERS. Its seasons were literally a series of random numbers. We *know* the team’s true quality, including injuries and surprises, was .560 the first year and .474 the third year, because that number is all each team was.
And that’s what Fangraphs’s simulations do. The editors do fantastic and detailed work to create the Depth Charts, which are best guesses. But once that guess is made, it becomes absolute. The Royals are no longer the Royals, they are an estimate of Runs and Runs Against; the Red Sox are no longer the Red Sox, the Nationals are no longer the Nationals. They are just numbers; the players made the numbers, but in the simulator, the players are gone.
In real life, the players are still there. They change. Thereby they change the real strength of the team. The simulator can’t know in advance what those changes will be. But the simulator could be programmed to expect that some sort of changes will happen. And then it would be less overconfident, and closer to truth.
Put another way: the 10,000 simulations ask “How often will a .510 team beat a .590 team over 162 games?”. They fail to ask “How often will we be wrong that it’s a .510 team versus a .590 team to begin with?”
…which you are proposing to address by adding random noise.
This is a circular argument. The times “a .590 team” loses more often than expected to “a .510 team” means the “.590 team” becomes something less than a .590 team. Winning percentage is a variable, not a constant. The constants are the probability weights for all possible outcomes, which are theoretically supposed to account for injuries, extreme performance deviations, etc.
I understand your argument conceptually, and why its so tempting, but its simply not sound. What you suggest would result in double-counting aspects on uncertainty due to perceived salience.
What you can argue is that the actual probability weights are flawed, perhaps that the tails of the distribution are fatter than accounted for, and attack the theoretical validity of the forecast model on those grounds. I haven’t looked in the black box (and even if I had, don’t know if I’d understand the technical mechanisms well enough to make a critical evaluation), so I can’t say whether such an argument can be made. But arguing that there salient uncertainty is somehow a separate concept from some other kind of “abstract” uncertainty is simply incorrect.
A D6? But that doesn’t account for Papelbon’s Gloves of Vorpal Choking, which require a D20 saving throw (after taking the appropriate modifiers for the enchanted familiar living in Harper’s beard).
11/10
One can project a teams record based on their true talent, but the standard error is 6 games either way, so a 90 win projection is really 84-96 with 1 SD confidence
Vegas had them winning 93 games and NYM winning 81.
Yeah screw you, Vegas! What’s the matter with those idiots anyway, don’t they know anything?
And having to hear about it deaily!
Well, they know they are trying to make money and not get the win total correct.
Correct! For over/under, spreads, etc. Vegas is only trying to have even betting on both sides so they win no matter what happens.
You guys better hope Rational Fan doesn’t see this thread.
Didn’t they also have the Padres at 83 wins? I need to go to Vegas.
Do we really give Harper a pass because he decides when he will hustle or not?? I saw him trot down to first on a routine ground ball that got bobbled months ago, so it’s not fatigue. This is a team game, not just a collection of talented individuals. The Cardinals had some serious injuries to key players and they might win 100 games. Excuse time is over.
Borderline incoherent.
Borderline? Only the first and second sentences even seem to have anything to do with one another.
Disagree. It was fully incoherent, a masterwork of disjointed half-thoughts.
If the ground ball got bobbled months ago, Harper should have at least made it to third.
Didn’t know Papelborn read Fangraphs.
I didn’t know Papelbon could read.
But all true. Congrats on not only being math experts but English majors too!! Fantastic!! Is it further to Chicago or by bus??
Read Ken Rosenthal’s article (or Jeff Passan’s). Numbers say Harper runs bases harder than anybody on his team. And if you watch the guy every day, you can’t possibly say he doesn’t play hard. Also in fairness, he was universally advised by media to “tone it down” for first few years of career after running into wall and playing like his hair was on fire. If you think harper doesn’t play baseball with intensity and is a loafer….then you don’t watch baseball.
I call it the “Dan Marino” complex. Remember, Dan Marino? Went to the Superbowl in his rookie year, and everybody of the time assumed he would go back many many times. Never happened again. He wrote many many records, played fantastic quarterback, having a pair of wideouts (duper and claton, aka the Marks brothers).
–Flash Forward to recent times–
Strasbourg arrives, everybody is all thinking good times ahead for the Nats. Harper on his heels. But, instead of chasing that pennant, the Nats powers that be, shut down Stras. Since that time, they haven’t had that “magic”, that aura, mystique, whatever you want to call it. They were supposed to ride Stras and Harper for a few championships. But, karma, the baseball gods frowning on Strasbourg being shut down, whatever, Stras has been less than advertised (still great, but…) and Harper? Poised to sign a monstrous contract ….. somewhere (else?).
Parallels? perhaps. When you have a chance, take it. There are no guarantees for the future.
It always amazes me how much trouble people have with the notion that a TEAM game has so much more going on that just the contributions of a few superstar players. Every sport in history is littered with all-time greats who didn’t collect much team hardware because any single player can only do so much … and yet failure is laid at their feet.
People are stoopid.
also…..Bryce harper is 22. LOL. i’m thinking he has a long time to win championships. And the lerners will probably sign him. The casual baseball world thinks they won’t but the casual baseball world is sort of uninformed. The lerners think of him as a son apparently and they have more money than any other owners. (see e.g. scherzer contract).
As a member of the enemy camp, a Mets fan, one of the problems with the Nationals down the stretch was their new closer and their displaced one. The Mets made several mid-season trade deadline moves that helped propel the team. Getting Johnson, Uribe & Clippard helped, not trading Tejada helped and of course stealing Cepedes really helped. The moves the Mets mead solidified the team. The BIG move by the Nationals bringing in Jonathan Papelbon, who approved the trade on the condition he came the closer. By the way the Nats has an effective closer at the time, Drew Storen. Now moved to set-up man Storen got beat by Cepedes 2 games in row in a pivotal series with the Mets and Papelbon was the loser in one of those games. The Papelbon trade undid the Nationals in the last quarter of the season. The trade didn’t help the bullpen, the clubhouse or team chemistry. The trade has made the front office and the manager look like idiots. Papelbon appeared to throwing his ego around, evidenced by the ugly incident with Harper. Papelbon didn’t have the cachet on that team the call out Harper, the hardest worker on the Nats, on anything in or out of the clubhouse! It the Nationals’ house was beginning to burn Papelbon was adding gasoline and a tank of oxygen to the fire. Now with house a pile of rubble and the Mets on top of the division and into the postseason and Nats will have to reload and rebuild.
Preseason projections are the slightly more scientific equivalent of preseason rankings in college football. Interesting, but essentially meaningless.
Actually, IIRC, both Nate Silver and Ken Pomeroy have found that preseason college rankings do add some predictive value over simply using prior year’s statistics. Not a huge amount, but enough to make it worth reflecting in a predictive model.
the Nats lost because the Mets played better ball. And did not enter the season thinking that they were entitled.
the “entitled” armchair psychology is ridiculous. You think the Nats didn’t play as hard because they thought they were owed the division (when they were full of free agents playing for new deals)? They just didn’t play as well. (although they have the same run differential, indicating a huge influence of luck and run sequencing). Not everything is about the evil mindset of the Nats. As somebody who watched them every day, they’ve had a tremendously talented roster (most wins in baseball from 2012-2014, but didn’t rise to challenge in the mostly crap shoot playoffs), but they one hundred percent were built up into a juggernaut that they weren’t by the national media. I don’t really think that’s their fault. And for everybody saying they should be blamed for expressing confidence in themselves, remember when Anthony Rizzo said the Cubs would win the Central? That was covered as endearing pluckiness, not arrogance. It depends on what you want the narrative to be. If Bryce Harper says “we’re going to win the east” that’s ridiculed by everyone who hates Bryce harper because he’s Bryce harper. They’re confident (rightfully so, they were an awesome team for 3 years, and it didn’t happen this year with injuries and underperformance. Let’s not ascribe it to some lazy hubris as opposed to just not executing/staying healthy/clutch play.