Skin in the (Ball) Game: Do Teams Underperform When They’re Out of the Race?

Recently, I was listening to one of my favorite non-baseball podcasts when baseball unexpectedly cropped up. Well, the theory of skin in the game cropped up. The idea, espoused by many people but notably by Nassim Taleb, is that actors perform better when they get rewarded for a good outcome and punished for a bad outcome. Want a better doctor? Fine them if they misdiagnose a patient, but give them a bonus for prescribing the correct treatment. Better money manager? Force them to invest all their own money alongside their client. You get the idea.

Anyway, one example of skin not being in the game is a sports team playing out the string. For most teams at most times, sports is a very skin-in-the-game-intensive field. If you hit well, you get paid more. If you don’t, you might get sent to the minors. If your team wins, they make the playoffs. If the team doesn’t win, no postseason. The incentives are straightforward.

At the end of a long season, however, it might not feel that way. If you’re 50-100 in late September, the rewards of a good game aren’t that high, and the cost of a bad game is quite low. If you’re 15 games out in the race, being 16 games out won’t suddenly bring out the detractors. You can think of these teams as having no skin in the game; the result of one game won’t change anything for them.

To some extent, you can think of teams that have already qualified for the playoffs as being in the same boat. If you’re 15 games up in your division, you might be more willing to experiment with a new pitch, or perhaps take it easy in the field to avoid injury. After all, winning or losing today matters less when tomorrow is already assured. That’s not to say nothing matters – playoff positioning is important, teams like carrying winning streaks into postseason play, and poor performance could lose a player his spot on the playoff roster – but a lot of the pressure is off.

That makes the teams in the middle, fighting for but not assured of a playoff spot, the ones with the most skin in the game. Winning is vitally important if you want to keep playing. Perform poorly, and that’s it. Doing well as a team is the only way to advance, which means your teammates will likely also hold you accountable.

Does all of this stuff matter? I could see it going either way. On one hand, I’ve seen bad teams sleepwalk through September with putrid records. On the other hand, the Pirates just swept the Dodgers. Bad baseball teams beat good baseball teams all the time, even if they seemingly have less to play for. I decided to conduct a study to see whether reality checks out with intuition.

The setup for this is fairly simple provided you have access to FanGraphs game and playoff odds for the last eight years, which I luckily do. I decided to focus on the last month of the regular season, because while we can call a team out of the race in the first month of the season, it probably doesn’t feel that way to players on the team. At 7-10, say, or 5-15, you might be unlikely to make the playoffs, but the lived experience won’t feel that way. By September, reality has likely set in.

I took our playoff odds for each team on each day of the last month of the regular season for the 2014-21 seasons. I grouped teams into three camps: likely out of the playoffs (less than 25% odds), in the hunt (25-75% playoff odds), and likely in the playoffs (more than 75% odds).

From there, I checked how each group did against each other group. For example, the teams that were likely out of the playoffs played to a .410 winning percentage against teams that were in the playoff hunt. That’s only 463 games; as it turns out, most games in September are played between teams that are both out of the hunt, and the other groups are somewhat less common.

How much should we expect those out-of-the-race teams to win? There are two ways of looking at this. I’ll detail them both before letting you know which I prefer. First, you could use season-to-date winning percentage for each team to come up with a naive win probability for the game. If a .600 team plays a .500 team, their expected win percentage should be .600. If they play a .400 team, their expected winning percentage should obviously be higher – .692, per Bill James’s Log5 approximation. We could even add home field advantage to clean the projections up a bit.

Next, you could use our projection-based odds. Those use the lineups each team submits, the scheduled starters, and a composite picture of each bullpen to create a team’s expected runs scored and runs allowed. Those are then converted to a winning percentage, and with a little home field advantage sprinkled in, you get projections for every game.

I’m partial to the second method. Using the first method, you could find that teams out of the race systematically underperform their season-to-date records and still not say anything about the skin-in-the-game theory. Why? Teams that are out of the race often run out weaker lineups on purpose. Want to see what you have in a rookie who blitzed Double-A at 21? Give him a shot! Want to give your star an extra day of rest, or skip your ace’s turn in the rotation for workload management? There’s no downside. Teams that are out of the playoffs often play weaker than their record, regardless of whether or not there’s some skin-in-the-game effect. They’re just using worse players!

Using the roster-aware odds accounts for that issue. By looking at the odds projected for each lineup, we can theoretically isolate how frequently a team should have won and compare it to what actually happened. That will give our answers more intellectual heft than just saying “oh they didn’t win too much, that seems bad.” Clearly, I chose to use our lineup-adjusted odds, though you could repeat the study with naive odds as well.

With our pregame predictions settled, I took every game contested in the last month of the regular season starting in 2014. For each game, I noted four things: the home team’s playoff odds coming into the day, the away team’s playoff odds coming into the day, the projected home team winning percentage, and which team won.

I then discarded games played between teams in the same group – two out-of-the-race teams can’t tell us much about skin in the game by playing each other, nor can two teams with similarly uncertain playoff futures; each team’s motivation is presumably equal. All we care about is what happens when one group plays another.

I’ll tell you my hypothesis: before I ran the numbers, I expected teams that were out of the race to underperform their expected winning percentages in every case. I also expected teams with their playoff berths secured to do slightly worse than expected against teams playing for their playoff lives, for similar reasons. Next, the results:

Ok, kind of what I said! But there’s standard error to consider. If you use a binomial estimation of standard deviation, teams that are out of the playoffs perform 1.4 standard deviations worse than you’d expect against teams in the hunt. In other words, they’ve won 15 fewer games than we’d expect out of the 463 contests they’ve played against those teams whose playoff destinies can change.

When out-of-the-race teams play against teams that have already secured their playoff berths, they don’t do as well as projected either. In fact, it’s a larger shortfall by standard deviation; out of the 1,319 times that down-and-out teams have faced off against playoff locks, they’ve won 37 fewer games than our odds predict. Finally, when teams that are already in the playoffs faced off against teams playing for their future, the locked-in teams comfortably outperformed their expectations.

The end takeaway of all this? I think you can say with some certainty that teams that are out of the playoffs play worse than their projected talent level down the stretch. Maybe that’s because they’re coasting, maybe that’s because their managers use lower-leverage relievers and generally protect their best arms, maybe it’s because they’re more likely to go out the night before and eat some fried food. I couldn’t hope to figure out why from this study. But the effect does feel real to me, even if it’s not a complete no-doubt lock.

There are a few other interesting effects – for example, this down-and-out penalty seems to apply more in road games than in home games, though the sample size is necessarily smaller there. Additionally, when down-and-out teams play each other in September, the home team massively outperforms expectations. Perhaps the crowd can keep teams in games that they’d otherwise give up on? Perhaps it’s less fun to stay out the night before when you have a chance to sleep in your own bed? Sounds like a fun thing to investigate sometime.

Overall, I’ll say this: I think that the skin-in-the-game effect exists in baseball. Teams that are out of the playoff hunt mostly play worse than you’d expect, even after you account for their diminished rosters. The exact composition of the effect is uncertain – and hey, I have a pile of data to play with, so if you have any other ideas for how to split this up, please let me know – but it seems very real.