How Much Did the Red Sox Benefit from Their Sign-Stealing Scheme?

Last week, the consequences of Houston’s sign-stealing scandal became clear after commissioner Rob Manfred announced the results of MLB’s investigation into the team’s “banging scheme” and use of replay review to electronically steal signs. By the end of the week, Houston’s field manager and general manager had been fired, and the collateral damage from that investigation led to two more managerial dismissals around the league. While the league-administered discipline for non-Houston personnel is still pending, everyone named in the report has been fired.

Now that Houston has been punished, Manfred will turn his attention to the Red Sox, who are under investigation for illegally using the replay room in their own sign-stealing system. Boston’s scheme wasn’t nearly as elaborate as the one used in Houston. Per a report from Ken Rosenthal and Evan Drellich in The Athletic:

Three people who were with the Red Sox during their 108-win 2018 season told The Athletic that during that regular season, at least some players visited the video replay room during games to learn the sign sequence opponents were using. The replay room is just steps from the home dugout at Fenway Park, through the same doors that lead to the batting cage. Every team’s replay staff travels to road games, making the system viable in other parks as well… The Red Sox’s system was possible only when a runner was on second base, or sometimes even on first base. Nonetheless, a team that is able to discern that information live, during a game, and relay it to base runners has a distinct advantage. A runner at second base can stare in at a flurry of catcher’s signs and know which one matters, then inform the hitter accordingly.

Rather than a using a trash can, the Red Sox decoded signs sequences in the video replay room and conveyed that information to the dugout. Once everyone knew the sign sequence, any runner on second base could communicate the signs to the batter via subtle movements or gestures.

The Red Sox will almost certainly face some sort of discipline, especially since they’re repeat offenders: In September 2017, both the Red Sox and Yankees received what now looks like a slap on the wrist for using electronic means to steal signs — on that occasion, using smart watches to communicate between members of the coaching staff and club personnel. Alex Cora obviously wasn’t part of the organization the first time the Red Sox were caught, but he was likely involved this time around. Cora and the Red Sox have already “mutually agreed to part ways,” but that won’t stop MLB from handing down some sort of suspension on top of the penalties it levies on the team.

From an on-field perspective, the limitations of the Red Sox system are immediately apparent. The system only works if there’s a runner on base to see the signs the catchers puts down. The threat of runners stealing signs from second base has always been a part of game — it’s the reason a different sign sequence is used once a runner reaches base — but decoding the sequence using video replay cuts out that gamesmanship.

As I was with the Astros, I was interested in seeing if we could decipher just how much the Red Sox benefited from their own sign-stealing system.

Back in November, I estimated the Astros cumulatively gained around five wins from their banging scheme. Would Boston’s less sophisticated system result in a similar cumulative benefit or would the constraints of requiring a runner to be on second base limit the net effect? The answer might seem obvious, but based on the reports, the Red Sox were able to use their system both at home and on the road, while the Astros were limited to using their cameras at home. That alone increases the sample size for the Red Sox to about half of the total pitches the Astros saw at home in 2017.

A simple look at the Red Sox’s wOBA with a runner on second reveals a big jump in performance in 2018:

At the surface-level, something seems amiss. The team’s improvement probably isn’t related to an upgraded roster, as it could have been with the Astros in 2017. Of the 11 batters who accumulated more than 200 plate appearances with the 2017 Red Sox, eight were back in 2018.

When we look at Boston’s plate discipline metrics with a runner on second, things look a bit murky:

As a team, with a man on second, the Red Sox chased pitches out of the zone far less often than in years prior but that didn’t necessarily translate to fewer swings and misses. And their contact rate on pitches in the zone actually fell from 2017 to 2018.

With that high-level look not very conclusive, let’s dive into the pitch-level data. As a refresher, I’m calculating run values for every pitch thrown using RE288 — the run expectancy based on the 24 base-out states and the 12 plate count states. When we filter and aggregate those run values, we can get a sense of how a team performed in particular situations, say, when a runner is on second base and they might be relaying the incoming pitch to the batter. To account for the different sample sizes, I scaled the run values to standardize the values per 100 pitches.

In 2018, the Red Sox saw a huge boost in performance when facing offspeed pitches with a runner on second base when compared to 2017, particularly when they swung at those pitches. They also saw a big improvement when swinging at fastballs. They saw smaller benefits when taking those two pitch types. It’s interesting that they didn’t see any improvement against breaking balls, though their improvement against the two other pitch types more than compensated.

What does the data show when we compare their 2018 performance with runners on second to their performance when there were runners on first or third but not on second?

Almost across the board, the Red Sox performed much better when there was a runner on second. And the biggest difference in performance was swinging at offspeed pitches like we saw above. In Rosenthal and Drellich’s initial report about the Red Sox sign-stealing scheme, there was some speculation that the signs could be stolen by a runner on first base. If that’s true, it doesn’t necessarily show up in the data. They clearly saw a benefit when there was specifically a runner on second.

Finally, let’s add their performance when the bases are empty to our analysis.

It’s clear that the Red Sox hit much better when there was a runner on second base. They simply crushed offspeed pitches and saw smaller benefits when facing breaking balls and fastballs. The obvious implication is that the team gained an advantage when a runner on second was able to relay the incoming pitch to the batter. When we compare their aggregate run values from 2017 to 2018, I estimate they gained a total cumulative value around five wins — the same estimated benefit the Astros saw with their sign-stealing scheme in 2017. And of course, that doesn’t completely account for the effects that are difficult to quantify, such as the cumulative benefit of requiring opposing pitchers to throw more pitches as hitters lay off of offspeed deliveries designed to entice a swing and a miss.

This isn’t the space to analyze how individual batters benefited from the system, particularly since no players were directly named in the report from Rosenthal and Drellich. I’d expect we’d see similar results to the Astros data though — marginal benefits for most individuals that add up to significant gains for the team. I was surprised to find that the total cumulative effect for Boston was on par with what I estimated for the Astros. It seemed like the limitations of Boston’s system would have resulted in a smaller effect. Perhaps the familiarity of receiving intel from a runner on second allowed Red Sox batters to effectively act on that information when presented; maybe it was hard for hitters to process information from a clanging trash can in the short time before they had to face an incoming pitch.

It appears that both sign-stealing ploys were similarly effective, which makes it’s clear that MLB needs to find a way to prevent teams from implementing these kind of schemes moving forward. Houston’s harsh punishment and the pending discipline for the Red Sox is one step, but the league needs to take more effective preventative measures. Further regulating who is allowed to use the replay room seems like a quick and simple fix that could be instituted as soon as this season. But as long as teams are looking for any and every small advantage and win-at-all-costs attitudes are promoted within organizational cultures, another controversy of this magnitude seems likely to happen again. Rather than react to that new disruption, MLB needs to be forward thinking, particularly as new technology affects how the game is viewed and played.