Our Playoff Odds Have Win Distributions Now
Much like the update last month, nothing in how we calculate our playoff odds has changed, but we’ve added a new feature to the page. We are now reporting the distribution of wins from the Monte Carlo simulation.
Specifically:
- The 25th percentile;
- The 75th percentile; and
- A histogram of projected wins.*
*The average is already reported as expected wins.
You can find this if you select the new display option in the dropdown of the same name.
During this winter’s iteration of the interminable debate about WAR, there was some discussion about reporting uncertainty in stats. Hopefully, providing these graphs of our playoff-odds output will help readers understand the playoff odds as something larger than a mere median estimate. It should be noted that we aren’t formally estimating error, just reporting the distribution of wins that have always been underlying the projections we show on the page. We don’t currently have error estimates for the projection systems that feed into the playoff odds.
The distribution is just the frequency of total win outcomes for a team’s 10,000 simulated seasons. The graph is scaled to be used within the same playoff-odds page, in order to compare a team’s range of outcomes to other teams. It shouldn’t be used to compare changes over different days or projection mode, because the differences in axes could make for a misleading comparison. For the curious, I’ve taken distributions from different days and projection modes and properly scaled the axes to demonstrate how they look in comparison to each other.
The Braves started 2018 with 72.9 projected wins and have improved that projection to 80.4 wins (entering May 9th). So the distribution has shifted to the right. I’ve also plotted the coin-flip distribution. It’s much like the the two FanGraphs projection distributions, except it will be shifted towards 81 wins. The season-to-date projection mode has a flatter distribution, because that projection mode adds random error terms to team’s game win probability.
As usual, please let us know if you have any problems or find any bugs.
I build things here.
Hot damn!
Woohoo!
Noice!
More histograms = less sad sadtrombone
Keep up the good work!
On a side note- would love to see an in-depth article on the National League off to a 30-17 record against the American League this season, this after many years of A.L. dominance.
Another fine addition!
Love it, but these ultimately still depends on the depth charts being accurate right? I know it’s impossible to predict how managers will behave in the future and injuries and everything, but sometimes I see things that I think most would agree not to not reflect current expectations of future playing time. Perhaps a standardized place where we could submit such feedback could help improve the accuracy of these projections.
Yea that’s my question too – does that Dodger’s forecast combine preseason expectations with what’s happened thus far record-wise only, or does it account for Seager missing the rest of the year?
It accounts of Seager missing the rest of the year.
Hot damn, that Yankees-Red Sox race.
So for projections, is the semi-interquartile range pretty much always 7 games?
It should narrow as the season progresses.
Hmm..I’m not seeing anything, and am getting all kinds of javascript errors in the console (running in incognito mode too). Maybe it’s just me for some reason, I’ll try again later but if you care:
playoffOdds-bundle.js?v=f6d8bb52ee3f4364d8ed:1 GET https://www.fangraphs.com/api/playoff-odds/odds?dateEnd=2018-05-10&dateDelta=&projectionMode=2&standingsType=div 500 (Internal Server Error)
(anonymous) @ playoffOdds-bundle.js?v=f6d8bb52ee3f4364d8ed:1
e.exports @ playoffOdds-bundle.js?v=f6d8bb52ee3f4364d8ed:1
e.exports @ playoffOdds-bundle.js?v=f6d8bb52ee3f4364d8ed:7
Promise.then (async)
r.request @ playoffOdds-bundle.js?v=f6d8bb52ee3f4364d8ed:7
r.(anonymous function) @ playoffOdds-bundle.js?v=f6d8bb52ee3f4364d8ed:7
(anonymous) @ playoffOdds-bundle.js?v=f6d8bb52ee3f4364d8ed:1
value @ playoffOdds-bundle.js?v=f6d8bb52ee3f4364d8ed:7
value @ playoffOdds-bundle.js?v=f6d8bb52ee3f4364d8ed:7
commitLifeCycles @ common-bundle.js?v=f6d8bb52ee3f4364d8ed:14
w @ common-bundle.js?v=f6d8bb52ee3f4364d8ed:14
C @ common-bundle.js?v=f6d8bb52ee3f4364d8ed:14
g @ common-bundle.js?v=f6d8bb52ee3f4364d8ed:14
v @ common-bundle.js?v=f6d8bb52ee3f4364d8ed:14
h @ common-bundle.js?v=f6d8bb52ee3f4364d8ed:14
s @ common-bundle.js?v=f6d8bb52ee3f4364d8ed:14
t @ common-bundle.js?v=f6d8bb52ee3f4364d8ed:14
updateContainer @ common-bundle.js?v=f6d8bb52ee3f4364d8ed:14
nn.render @ common-bundle.js?v=f6d8bb52ee3f4364d8ed:22
(anonymous) @ common-bundle.js?v=f6d8bb52ee3f4364d8ed:14
unbatchedUpdates @ common-bundle.js?v=f6d8bb52ee3f4364d8ed:14
ln @ common-bundle.js?v=f6d8bb52ee3f4364d8ed:14
render @ common-bundle.js?v=f6d8bb52ee3f4364d8ed:22
(anonymous) @ playoffOdds-bundle.js?v=f6d8bb52ee3f4364d8ed:7
t @ common-bundle.js?v=f6d8bb52ee3f4364d8ed:1
window.webpackJsonp @ common-bundle.js?v=f6d8bb52ee3f4364d8ed:1
(anonymous) @ playoffOdds-bundle.js?v=f6d8bb52ee3f4364d8ed:1
playoffOdds-bundle.js?v=f6d8bb52ee3f4364d8ed:1 Uncaught (in promise) Error: Request failed with status code 500
at e.exports (playoffOdds-bundle.js?v=f6d8bb52ee3f4364d8ed:1)
at e.exports (playoffOdds-bundle.js?v=f6d8bb52ee3f4364d8ed:7)
at XMLHttpRequest.h.(/standings/playoff-odds/fg/anonymous function) (https://www.fangraphs.com/dist/playoffOdds-bundle.js?v=f6d8bb52ee3f4364d8ed:1:25467)
It’s an intermittent bug that I’ve been trying to track down. Thanks for letting me know!
Hey, this is a great addition! Sean, I’m hoping that next you can fix the combined war leaderboards. They currently only show PA or innings pitched, so for instance NL pitchers don’t have any PAs, and Shohei Ohtani doesn’t have any innings pitched.
Thanks!
Those are going to be overhauled somewhat soon, we’ll definitely change that then. The last (minor) change we made to them was just a stop gap, since we didn’t have the WAR values added together anywhere else on the site.
Sean, these look terrific! Great to see this. Two thumbs up for the great care taken in discussing what potential error is and isn’t reflected here.
Thanks, Jonathan!
“It should be noted that we aren’t formally estimating error, just reporting the distribution of wins that have always been underlying the projections we show on the page. We don’t currently have error estimates for the projection systems that feed into the playoff odds.”
Is this basically why the distribution of wins is similar to the distribution in coin-flip mode? Like, I’d expect a wider distribution for the projections, since you have more uncertainty in the true talent of a team. Like you know a coin is exactly a .500 team, but a team projected .500 has it’s own distribution of what it could actually be.
So darn civilized! Stats really, really needs to do better incorporating data variability. What’s the distribution of data look like? What are the error bars? All sorts of questions. Beautiful stuff!
I think you must mean that the coin flip since May 9 has shifted towards 20+(162-(20+14))/2 = 84 wins.
The Astros 25th percentile Wins is 96? Whoa.
After the Phillies acquired Jake Arrieta, I said to my friend “If there’s a bell curve around a team’s potential outcomes, Arrieta moves the right end of the bell curve into wild card contender territory.”
To see this visualized is awesome. Thank you.
Braves are in first!