Looking Back at the 2022 ZiPS Projections

© Charles LeClaire-USA TODAY Sports

Before we get to the 2023 ZiPS projections, there’s still some unfinished work from 2022 to do. Looking at which projections went the most wayward is definitely an exercise that makes me soul cringe a bit, but in any model, being wrong is an important component of eventually being right. Calibration is a long-term project, and while chasing greater accuracy in mean projections isn’t likely to result in any huge bounty — there’s a reason projection systems are so tightly clustered — there’s still improvement to be had in things like calibrating uncertainty and long-term data.

Let’s start with how teams performed versus their projections:

2022 ZiPS Projected Wins vs. Actual Wins
Team ZiPS Real Difference
Baltimore Orioles 64 83 19
Los Angeles Dodgers 93 111 18
Houston Astros 90 106 16
Cleveland Guardians 78 92 14
New York Mets 88 101 13
New York Yankees 88 99 11
Atlanta Braves 90 101 11
Seattle Mariners 85 90 5
St. Louis Cardinals 88 93 5
Philadelphia Phillies 83 87 4
Toronto Blue Jays 88 92 4
Arizona Diamondbacks 71 74 3
San Diego Padres 89 89 0
Milwaukee Brewers 87 86 -1
Tampa Bay Rays 88 86 -2
Colorado Rockies 70 68 -2
Chicago Cubs 77 74 -3
San Francisco Giants 85 81 -4
Kansas City Royals 70 65 -5
Pittsburgh Pirates 68 62 -6
Chicago White Sox 88 81 -7
Detroit Tigers 73 66 -7
Minnesota Twins 86 78 -8
Los Angeles Angels 81 73 -8
Oakland Athletics 68 60 -8
Texas Rangers 77 68 -9
Boston Red Sox 88 78 -10
Cincinnati Reds 74 62 -12
Miami Marlins 82 69 -13
Washington Nationals 76 55 -21

Teams have gotten a bit more polarized in how they’re run in-season. Looking at the in-season ZiPS projections, roster strength has varied much more in recent years than when I started doing this. It wouldn’t be surprising to see the mean absolute error — for an exercise like this, I want to use the simplest tool that gets the point across — creep up over time. That is the case here, as the MAE of 8.3 wins is above the ZiPS historical average of 7.5 (not including 2020). ZiPS underperformed its usual matchup vs. Vegas, only going 17-13 in over/unders as of the date of release (April 6); historically, ZiPS has averaged 19-11.

What would a perfect projection look like? If you knew the win probability of every game played before it was played, the MAE of your seasonal projections, based on the hypergeometric distribution, would be around five wins.

Thirty-six percent of the estimated error in ZiPS was simply guessing wrong on roster construction. With perfect knowledge of who actually played, the MAE drops to 7.1 wins. The biggest gains in accuracy would have been the Boston Red Sox (88 wins projected, 78 knowing plate appearances/total batters faced), the New York Mets (88, 96), and the Cleveland Guardians (78, 84). Amusingly, the Orioles were one of the four teams ZiPS would have actually missed on by a larger amount if the system had known playing time information ahead of time. Simply put, ZiPS would have been even more negative about the O’s if it had known what their rotation would look like!

With 10 deciles and 30 teams, you ideally want three teams to finish with a percentile projection in each decile. In other words, you want three teams to fail to meet their 10th-percentile projection, three teams to at least hit their 90th-percentile projections, and so on. So how did ZiPS do?

2022 ZiPS Projection Deciles
Decile Teams
First 3
Second 3
Third 5
Fourth 3
Fifth 3
Sixth 1
Seventh 3
Eighth 2
Ninth 2
Tenth 5

Amusingly, ZiPS got four of the six divisions in the right order, with the AL Central error being Cleveland actually finishing first instead of third, and a real mess in the AL East (four teams projected at 88 wins and the O’s were the biggest out-performers in baseball). There was no change in the year-to-year correlation between errors; there’s no significant relationship (an r-squared of -0.0013) over the entirety of ZiPS’ history. This is also true on a per-franchise level. What this means is that there’s no tendency for errors one year to predict errors the following year. This holds true even on the team level, with the Mariners having the highest r-squared in season-to-season errors at 0.02. So, no, there’s no reason to think the 2021 Giants uncovered a special ability to outperform projections; they merely outperformed projections in 2021.

For hitters and pitchers with at least 300 plate appearances or batters faced, the MAE for OPS+ was 16.3 points, while for ERA+, it was 20.9 points. Both of these are consistent with the results over the last decade (16.4 and 21.1, respectively). I actually expected to do a little worse than normal.

Here are the hitters the system most underestimated:

Biggest ZiPS Hitter Misses (Positive)
Name OPS+ Projected OPS+ Difference
Albert Pujols 154 80 74
Aaron Judge 211 144 67
Paul Goldschmidt 180 120 60
Michael Harris II 135 75 60
Andrés Giménez 141 93 48
William Contreras 138 94 44
Brandon Drury 122 78 44
Jose Altuve 160 116 44
Christopher Morel 107 64 43
Brendan Donovan 126 83 43
Jake McCarthy 118 76 42
Joc Pederson 144 102 42
Nolan Arenado 154 116 38
Yordan Alvarez 187 150 37
Cal Raleigh 122 85 37

I believe that the miss for Albert Pujols is the largest ever in ZiPS for a player 35 years or older. And I am very happy to have missed on that one! I think it’s unfortunate that we have a player who is a slam-dunk, first-ballot Hall of Famer, but who basically spent the last decade of his career hitting like Darin Ruf instead of Jimmie Foxx. There are a lot of younger baseball fans whose only experience of Pujols has been that of a mediocre 1B/DH. While 2022 wouldn’t challenge any of the classic seasons from Pujols’ first decade of play, getting one last burst of glory was really cool to see and lifted his farewell season above the sad trudge of Derek Jeter’s endless goodbye in his final campaign.

The unexpectedly excellent performance of Michael Harris II had a greater impact on his future projections than any of the other top performers here. Given his young age, there was a lot of uncertainty about his upside. He was actually considerably more likely to hit his actual OPS+ (135) than any of the other 14 players listed. Now that he’s apparently hitting that upside, his 2023 ZiPS projection has seen a massive bump. With only 154 plate appearances, Matt Carpenter did not make this list, but he was the most unexpected player overall in 2022, with the projections only giving him a one-in-2,300 chance of hitting as well as he did, even in a relatively small sample.

Now here are hitters who most underperformed their projections:

Biggest ZiPS Hitter Misses (Negative)
Name OPS+ Projected OPS+ Diff
Yasmani Grandal 64 128 -64
Joey Gallo 79 131 -52
Spencer Torkelson 77 122 -45
Leury Garcia 42 85 -43
Jared Walsh 81 124 -43
Franmil Reyes 81 124 -43
Max Muncy 96 135 -39
Yoán Moncada 76 115 -39
Jonathan Schoop 62 101 -39
Enrique Hernández 75 113 -38
Cody Bellinger 78 115 -37
Avisaíl García 65 101 -36
Abraham Toro 63 98 -35
Tyler O’Neill 101 133 -32
Andrew Velazquez 53 84 -31

Three White Sox made this list, with two of those three expected to be among the team’s most valuable players heading into the season (I’ll let you guess the odd man out). It’s not surprising Chicago’s offense struggle to score runs in 2022. One interesting thing from the larger dataset, which you can see in some of the names above, is that being a Three True Outcomes player had a slight effect on projection accuracy in 2022, one that hadn’t been present at any time in the past. I’m definitely curious to see if this development persists, assuming we have the same deadened baseballs we appeared to have this season. Now let’s turn to the pitchers:

Biggest ZiPS Pitcher Misses (Positive)
Pitcher ERA+ Projected Difference
Matt Moore 203 79 124
Brock Burke 200 94 106
Tony Gonsolin 196 94 102
Justin Verlander 220 128 92
Jaime Barría 154 78 76
Tyler Anderson 163 92 71
Dylan Cease 180 113 67
Julio Urías 194 131 63
Nestor Cortes Jr. 159 98 61
Clayton Kershaw 184 127 57
Erasmo Ramírez 134 79 55
Sandy Alcantara 178 123 55
Spencer Strider 152 100 52
Shohei Ohtani 172 122 50
Adrian Sampson 132 83 49

I think half of the Dodgers starting rotation made this list, which is hardly surprising given that the Dodgers outperformed their already excellent projection by only one fewer win than the Orioles did. Andrew Heaney wasn’t far off from making this group, either, and that’s with ZiPS already being more optimistic about the left-hander than most. If you bet on Matt Moore being a highly valuable relief pitcher, pat yourself on the back; ZiPS sure didn’t! Now for the under-performers:

Biggest ZiPS Pitcher Misses (Negative)
Pitcher ERA+ Projected Difference
Dallas Keuchel 44 99 -55
Trevor Rogers 74 125 -51
Lucas Giolito 81 131 -50
José Berríos 74 117 -43
Patrick Corbin 62 105 -43
Gerrit Cole 111 154 -43
Ian Anderson 81 121 -40
Charlie Morton 94 131 -37
Alex Wood 79 116 -37
Sean Manaea 75 112 -37
Mike Clevinger 86 121 -35
Aaron Ashby 89 120 -31
Joan Adon 55 85 -30
Eduardo Rodriguez 93 123 -30
Mike Minor 74 100 -26

Hey look, more White Sox! Given how poorly so much of their team did relative to the projections, I’m actually a little surprised that ZiPS only over-projected them by seven wins (88 vs. 81). I agreed with ZiPS that Dallas Keuchel would at least be a serviceable fifth starter this season, which absolutely did not happen! That said, I didn’t agree with ZiPS’ relatively friendly projection for Patrick Corbin, so take that, set of algorithms I developed!

Let’s finish up with the calibration data for pitchers and hitters. This is actually something I focus on a lot more than the mean projections. Models aren’t right, they’re useful, and a properly calibrated projection system should know how often it is expected to be wrong.

2022 ZiPS Projection Deciles
Decile ERA+ OPS+
First 10.7% 8.3%
Second 10.1% 10.5%
Third 8.4% 9.4%
Fourth 11.8% 10.5%
Fifth 11.2% 9.8%
Sixth 10.7% 11.2%
Seventh 9.6% 8.7%
Eighth 8.4% 10.5%
Ninth 7.9% 12.0%
Tenth 11.2% 9.1%

Next up: the comically preliminary ZiPS projected team standings for 2023!





Dan Szymborski is a senior writer for FanGraphs and the developer of the ZiPS projection system. He was a writer for ESPN.com from 2010-2018, a regular guest on a number of radio shows and podcasts, and a voting BBWAA member. He also maintains a terrible Twitter account at @DSzymborski.

41 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
JohnThackermember
4 days ago

“ZiPS underperformed its usual matchup vs. Vegas, only going 17-13 in over/unders as of the date of release (April 6); historically, ZiPS has averaged 19-11.”

You might expect that over time eventually the smart money would use ZiPS to bet against the dumb money in Vegas and tighten the odds. (It’s actually not expected profit maximizing for the Vegas bookies to set the correct lines if it doesn’t balance the money.) I wonder if there’s any trend to that as sabermetrics becomes more well known by sections of the betting public.

airforce21one
3 days ago
Reply to  Dan Szymborski

Kind of like not sharing the career ZiPS player projections…