I Don’t Believe in the Cleveland Guardians* by Ben Clemens September 28, 2022 © Jerome Miron-USA TODAY Sports Oh, did that title get your attention? I thought it might. Bad news, though! It was just a trap to get you to read this. I’m here to talk about the same thing we talk about around this time every year: projections offending people. I don’t like it any more than you do, but that’s just the name of the game when October comes around. We post playoff odds before the season, which means we’re always missing on some team or other. That’s right: as best as I can tell, Cleveland fans are upset that we gave their team a 93.3% chance of making the playoffs in 2019, only to have them miss out on the postseason. Okay, fine. I’m actually talking about the Guardians making the playoffs this year after starting the season with a 7.5% chance of winning their division, as the league helpfully noted on Twitter earlier this week: That's why they play the games. ? pic.twitter.com/X0KkQwgPr1 — MLB (@MLB) September 26, 2022 I brought up the 2019 example to make a point: our odds miss in both directions. They’re not biased for or against the Guardians specifically. I thought it might be useful to look into a few things our model doesn’t handle particularly well that might have understated the Guardians’ chances, a few things the team did well to improve its odds, and a few breaks along the way that Cleveland deftly took advantage of. Depth The Guardians came into the year with question marks across their lineup. Our Depth Charts projections did their best, but they were simply a hard team to peg. We allocated roughly 2,200 plate appearances to Franmil Reyes, Bobby Bradley, Yu Chang, Oscar Mercado, Gabriel Arias, and Bryan Lavastida. That uninspiring sextet has combined for 474 plate appearances. Four of the six played their way off the team (Mercado is back after a brief cameo in Philadelphia). We weren’t particularly high on any of those players, which helped weigh down Cleveland’s offensive projections – that’s a lot of plate appearances to give to bad hitters. The Guardians clearly concurred. What we don’t account for in our methodology is their depth. In essence, the team threw a bunch of near-major-league-ready players at the wall and saw what stuck. Oscar Gonzalez has been solid, so he jumped into a starting role. Steven Kwan and Andrés Giménez have excelled, and earned more playing time as a result. Owen Miller has seen his role increase, though he hasn’t exactly run with it. It didn’t have to be these exact players, either; if Arias had been an unexpected gem instead of Gonzalez, the team would have adjusted and found a way to play him instead. This is a fundamental limitation of the way we calculate our odds. Because we use point forecasts, we can’t account for the effect of in-season volatility. Say you have three players at the same position, and your best guess is that each of them has a true talent of 2 WAR/600. Our playoff odds treat that simply: one of them is a starter, one is a backup, and one doesn’t get assigned any playing time. In reality, though, you can give each a tryout. You might think they’re each 2 WAR players, but by getting more in-season information, you can improve your estimate. Players are better or worse than we expect all the time. Having a ton of guys for one spot actually improves the expected production for that spot, because you never know when one of them might pop. The age and experience of these players matters, too. To pick a divisional example, I think our model does a pretty good job of forecasting second base for the White Sox because I just don’t think there’s much uncertainty about the true talent level of Josh Harrison and Leury Garcia. You mostly know what you’re getting with both guys. The less experienced the player, however, the bigger the error bands. That might lead to more complete busts – Arias hasn’t looked ready for the majors in a few cups of coffee – but it also leads to more players we underestimated. Bullpen This is basically just a miss. I don’t feel particularly bad about it; the Guardians had a middle-of-the-pack bullpen last year and lost a few guys. We thought they’d have another mid-table finish there, but we failed to anticipate all kinds of interesting names. I have a fascination with Cleveland’s relievers. They’ve been lights out all year. We have them as the fourth-best relief unit in the majors, and I wouldn’t be too upset with you if you think they’re better than that. What is there to learn here? Not much, at least to my eyes. Did you know that bullpens are hard to forecast? Did you know that Emmanuel Clase is awesome? Did you know that Trevor Stephan would turn his splitter from a novelty into his best pitch? Relievers are volatile, our projections of them are necessarily imperfect, and trying to predict each year’s breakout bullpen isn’t something you accomplish with these types of models. If anything, I’d want our forecasts to regress further towards the mean, which would have missed on the Cleveland bullpen anyway. Opposition As much as our projection methodology misses deep teams, its greatest shortcoming is the opposite: giving shallow teams too much credit. This is another inherent limitation of our playoff odds, one best illustrated by the White Sox. The opposite of giving a team too little credit for their high-volatility depth pieces is giving a team too much credit for their stars. That’s less about talent level and more about availability, though of course talent level is never guaranteed. For example, we pegged Luis Robert, Eloy Jiménez, and Tim Anderson for more than 600 plate appearances apiece. Through Monday’s action, the highest any of them has reached is Robert’s 401, and he played through injury to get there. In many seasons, everyone might remain healthy, but certainly not in every season, and the White Sox have had a rough run of injuries. (The Guardians, meanwhile, have been remarkably healthy.) It isn’t even limited to those three; Yoán Moncada has been both injured and disappointing, which leads me to another issue in our forecasts: what happens when a player unexpectedly declines. If that player is someone like Reyes, teams can do what the Guardians did, namely replace him with a similar bat and move on, with little loss to the projections. If the backup is a lot worse – sorry, Seby Zavala and to a lesser extent Jake Burger – any loss of production or availability from a star hurts more. The White Sox had a gleaming preseason projection in large part because of their star-studded starting lineup. Instead, they’ve relied on a heaping helping of Harrison, Garcia, Andrew Vaughn, Gavin Sheets, AJ Pollock, and Adam Engel. There are a few successes in that group, but not enough to replace what they’ve been missing. Knock the White Sox down by five or so wins, and the Central would have looked much closer. When we use a single estimate for playing time, rather than randomly creating injuries in each simulation, we give teams with a strong starting lineup or rotation but little depth too much credit. Things don’t always go wrong, but they often do. Backup plans matter. Those three factors undoubtedly lowered the Guardians’ preseason odds. The depth-related ones are a fact of life given the way we simulate the season. We’ve looked into various workarounds, but they detract from one thing that we think is great about our model: its simplicity. There aren’t a lot of moving parts. We take projections and depth charts, use those to estimate team strength, and simulate the season a bunch of times. If I were trying to beat the market and make a living betting on baseball, I’d try to improve on our methodology. But for a system that runs at the conclusion of every game, all season long, I’m quite happy with what we have. Maybe you could pinch in the ends of the distribution – our methodology tends to underestimate long shots slightly and overestimate favorites – but we’re talking about pretty small numbers here in the aggregate. Heck, even if you did want to improve on our methodology, pretty much every baseball prognosticator thought the Guardians were unlikely to win the AL Central. Baseball Reference had them just over 10% to win before the season started (and 9.1% on July 13, by which point we had them up around 20%). Baseball Prospectus pegged them for 78 wins. Per an ESPN roundup, betting odds made them 8-1 underdogs to win the division, which works out to roughly 12% after accounting for house edge. Put another way, the Guardians needed to have some things go right for them this year. They’ve done just that, and more power to them. Kwan, who appeared on few preseason top 100 prospect lists (though he did on ours!), is a key cog in the offense. The team shuffled through options until it found a solid starting lineup, and Terry Francona got the best out of a talented bullpen. They also caught a break from their divisional foes – the AL Central is the lone division with only one team above .500, and even then the Guardians have the worst record of any division leader. A feel-good story of a young team arriving sooner than expected is cool, and that’s exactly what this is. Oh yeah, one more thing. This one is admittedly just me being annoyed. Maybe you saw this tweet: FanGraphs had the @CleGuardians <50% to win their division only 15 days ago! 100% chance you don't want to face them right now. https://t.co/IJbrR2Fhrt — Codify (@CodifyBaseball) September 26, 2022 First, it’s not factually accurate, though that doesn’t bother me too much; the Guardians were actually 67.7% likely to win the division per our playoff odds. That’s not particularly close to 50%. That’s just a miscounting of days, though. Move back three days to September 8, and we had them 43.8% to win the division. What great disrespect! Only, not really. Why were their odds what they were? Because the White Sox and Twins were each only 1.5 games back. Meanwhile, the Guardians still had to play eight games against the Twins and four against the Sox. In retrospect, the Guardians won 10 of those 12 games and pulled away. But ex ante, with a slim lead over two divisional foes and plenty of games remaining against both, is it any wonder the Guardians were roughly 50/50 to win the division? They had higher odds than either of the teams chasing them, as befits their lead in the standings. In essence, all of these “the models hated this team and now they’re winning their division two weeks later” claims have the same flaw: basically without fail, the team in question goes on a tear in between the “shocking” model prediction and the time someone brings it up. It’s a question of arbitrary endpoints. The Guardians, for example, had slumped to get to that point; they’d been 72.8% favorites to win the division in late August before a 3-7 skid. Since September 8, they’ve gone 16-2. That’s pretty good! I doubt a lot of Guardians fans thought 16-2 was the most likely outcome of those 18 games. Go 16-2, with 12 of those 18 coming against the two teams just behind you in the standings, and your playoff odds will improve quite a lot. It’s true that no one wants to face them in the playoffs, because no one wants to face anyone in the playoffs. Playoff baseball teams are good! That’s how they got there. There are no walkovers between good teams in five-game series, never mind three-game series. That’s just the nature of baseball. Everyone is dangerous. I’d have a lot more sympathy for these claims, with their implied “your model sucks,” if they weren’t invariably made after the fact, and almost always after huge winning streaks. Did you know that going 16-2 improves your chances of making the playoffs markedly? Shocking! “No one believed in us” is a great rallying cry. It’s meaningless at this point, though. The New England Patriots, the greatest American sports dynasty of the 21st century, were using it eight years ago, in the middle of their peak. Clemson’s football team is simultaneously one of the three great powers in the sport and, if you believe coach Dabo Swinney, an outfit that no one believes in. Duke basketball? Join the club. LeBron James, maybe the most believed-in player in the world, is no stranger to the cry. “Us against them” is a powerful psychological bonding tool; an actual lack of belief isn’t required. So, should Cleveland fans feel aggrieved by our projections? I’m biased, but I don’t think so. The Guardians have played extremely well so far this year. They’ve manufactured their own luck in addition to receiving some from the rest of the division. Their jam-packed 40-man roster paid dividends this year, providing them with multiple options across the diamond and replenishing their offense when previous key contributors stumbled. Let’s not diminish their accomplishment by invoking some imagined slight and making the story about preseason odds instead of an impressive season.