Why Are the Orioles’ Playoff Odds So Low?

At this point, it’s becoming a meme. The Orioles chug along, at or around .500, and our playoff odds continue to say that they’ll almost certainly miss postseason play. Across the internet, sites like Baseball Reference and FiveThirtyEight give them a higher chance. The headlines write themselves: “Why doesn’t FanGraphs believe in the Orioles?”

Just to give you an example, after the games of July 29, the Orioles were 51–49. Baseball Reference gave them a 34% chance of reaching the playoffs; we gave them a 4.6% chance. Ten days later, on August 8, Baseball Prospectus pegged them at 22.2% while we had them at 5.4%. On August 11, FiveThirtyEight estimated their playoff odds at 16%; we had those odds at 5.7%. Another week later, on August 19, Baseball Reference pegged them at 35.5% to reach the playoffs; we gave them a 4% chance. You can snapshot whatever day you’d like and you’d reach the same conclusion: we don’t think the Orioles are very likely to make the playoffs, while other outlets do.

Now, we’re getting down to brass tacks. The Orioles are 68–61 after Wednesday’s games. Baseball Reference thinks they are 43.6% to reach the postseason. FiveThirtyEight isn’t quite so optimistic, but still gives them 23% odds, while Baseball Prospectus has them at 29.9%. Here at FanGraphs, we’re down at 6.6%, even after they called up top prospect Gunnar Henderson. Why don’t we believe?

The answer comes down to the way we construct our odds. Our methodology is time-consuming but fairly straightforward. At its core, it relies on the two projection systems we use on the site: ZiPS and Steamer. We take updated rest-of-season projections for every player in baseball, as well as rest-of-season playing time projections from RosterResource. We stitch those together into estimates of a team’s overall line on both sides of the ball, then use BaseRuns to convert those lines into runs scored and allowed per game. Finally, we use a Pythagorean estimation to estimate a true talent level for that team. Repeat this process for every team in baseball, and it gives us the matchups for every remaining game in the regular season.

That’s the core of our disagreement with other outlets about the Orioles’ playoff odds: we don’t think they’re as good as their record. Mostly, that comes down to pitching. So far this year, the O’s have scored 4.21 runs per game while allowing 4.13. We think their offense will actually get better down the stretch, scoring 4.33 runs per game. The pitching fares far worse, though: we think they’ll allow 4.91 runs per game, which places them squarely below .500 in terms of expected record — .437, to be precise.

Those forward-looking odds take what teams have done this year into account, but they also take previous performance into account. Take Nolan Arenado, for example. So far this year, he’s hitting .307/.370/.571, good for a 163 wRC+. You could project that to continue for the rest of the year, but our projections don’t because they look at his prior career line as well. Instead, we project him to hit .273/.337/.494 the rest of the way, a still-excellent 132 wRC+. That offensive gap means we project the Cardinals to get less from him for the rest of the year than they have so far.

Baseball Reference uses a simple methodology — literally. They use Simple Rating System with some mean reversion thrown in, detailed here. In essence, they look at how the Orioles have performed in their past 100 games, add in 50 games of .500 baseball, and use that as their projected winning percentage. That’s quite similar to our Season-to-Date odds method, which sees the Orioles as slightly better than .500 the rest of the way; those odds have the Orioles with a 40.2% chance of reaching the playoffs this year. Their method is the easiest to replicate, so I’ll focus on that today.

That’s the central question to be answered in any consideration of the Orioles’ playoff odds. Should we focus exclusively on what they’ve accomplished this year, or take each player’s prior baseball history into account? There’s one further complication: both our Season-to-Date mode and the SRS method ignore personnel changes. They don’t know that the Orioles got better in June when Adley Rutschman was called up (though he’s been up long enough that his contributions are fairly well baked into the pie at this point), but they also don’t know that the team traded Jorge López and Trey Mancini at the deadline. That’s on purpose: the beauty of those two methods is that you can replicate them easily, no opaque projections necessary.

That raises an interesting question: why do our projections think that the Orioles pitching staff will perform worse than it has so far this season for the remainder of the year? If they keep pitching like they have, our odds are too low. To try to explain what the projections are doing, I’ve prepared a simple chart. Here are the top 10 Orioles pitchers’ ERAs and FIPs this year (using RosterResource to pick the top 10), as well as our ERA projection the rest of the way:

Other than Kyle Bradish, everyone is projected to get worse. Mostly, that’s because they’d put up poor performances before this year in the majors. For rookies, their minor league numbers didn’t herald a breakout. If you believe in the projection systems, Orioles pitching has been playing over its head. That’s not to say it can’t continue to do so, but I think systems are right to be skeptical that an entire unit can continue to show a heretofore unseen level of performance.

Another way of thinking about it is that there have been plenty of times over the past eight years (that’s how long we’ve run our projections in their current form) where our season-to-date odds and projection-based odds have regarded teams differently. Which one has done better? We can test them using a “gambling” method that I used earlier this year.

My method: take each team’s playoff odds on every day of August for the 2014–21 seasons, excluding 2020. I used August to highlight the time of the year we’re most interested in here, when plenty of games have been played, but we’re not quite to an endgame scenario yet. No one’s interested in what each system says when there are three games left in the season, or when there are 130; it’s right now, when we’ve seen a lot of the season but there are still plenty of games on the docket.

From there, I had each system wager against the other system for every team, every day. Let’s use a hypothetical example: say that on a given day, the projections mode gave a team a 5% chance of making the playoffs, and the season-to-date mode gave them a 25% chance of making the playoffs. The projections mode would “bet” against the team making the playoffs, with that 25% chance from the season-to-date model forming the odds. If the team subsequently missed the playoffs, I’d credit the projections model with a positive 0.25 score. If the team made the playoffs, I’d debit them 0.75. Likewise, the season-to-date model would “bet” on the team making the playoffs. They’d lose 0.05 if the team missed the playoffs or make 0.95 if the team made the playoffs. I did that for every game in August, every year other than 2020, starting in the first year of our current methodology, 2014.

If our model is systematically messing things up in August, you’d expect the season-to-date mode to rack up a hugely positive score by consistently fading the excesses of the projections. Instead, the results were lopsided in favor of the fancier, projection-aided model. Over 6,510 projections (one per team per day of August), the projection model posted a score of +282. The season-to-date model posted a score of -25. In other words, if you see a disagreement between the two modes, you should lean in the direction of the projections, even if you don’t use them as gospel.

Maybe we’re not interested in August as a whole, though. Does anyone actually care that the Cardinals are 97.3% likely to make the playoffs based on projections and 99.6% likely to make it based on season-to-date statistics? Not really. I ran a separate cut — every day where the two methods differed by at least 20 percentage points.

I expected both modes to put up positive numbers here, because when two things are that far apart, the likeliest true outcome seems to be in the middle. I wasn’t disappointed; across 4,920 observations, the projections system accumulated a score of +940, and the season-to-date system accumulated a score of +502. Another way of thinking about that: our projection-based odds are generally picking up on something useful when they differ from simple season-to-date odds, which explains their high score. They’re too confident, though. They move too far from the baseline, which explains the smaller positive score achieved by betting against them.

For completeness’s sake, I also looked at only the times where projection-based odds were 20 percentage points lower than season-to-date odds, rather than when they were lower or higher by that much. Is there something specific going on when we think a team is unlikely to make the playoffs? Nope — the projections racked up a two-to-one score advantage, +387 to +190.

This has all gotten rather long-winded, but let me give you the high points. Our odds give the Orioles a lower chance of reaching the playoffs than some other popular sites. That’s because we bake in pre-2022 performance in making projections, rather than just their run differential this year. Over time, our odds have done pretty well, both late in the year and when they differ markedly from season-to-date-style odds. If you wanted to be cute, you could get an even better estimator by using two-thirds projection-weighted odds and one-third season-to-date odds, but that sounds a lot like overfitting to me. That would give the O’s a 17.8% chance of making the playoffs, which probably wouldn’t do much to placate fans.

Truthfully, I’m not surprised by this outcome. When I looked into our odds’ accuracy last year, I found that our odds were better than season-to-date odds even late in the season. I do think we’re too harsh on the Orioles; the true answer is likely somewhere in between the two. We tend to underestimate slightly teams with a low chance of making the playoffs; of the 2,201 observations with between 5–10% chances of making the playoffs, we predicted 7.4% would make the playoffs when in reality 11.5% did. Combine that with the fact that our method’s edge over season-to-date models fades late in the season, and it’s reasonable to think we’re low on the Birds.

But only a little low! For the most part, our odds do a pretty good job of estimating how hard the road ahead is for teams. That doesn’t mean, by any stretch of the imagination, that the Orioles won’t make the playoffs. All it means is that they have their work cut out for them. Plenty of players on the Orioles have exceeded expectations so far this year. If they manage to keep that up and make the playoffs, they will have defied the odds — whichever set you choose to use.