How Much Did the Red Sox Benefit from Their Sign-Stealing Scheme?

Last week, the consequences of Houston’s sign-stealing scandal became clear after commissioner Rob Manfred announced the results of MLB’s investigation into the team’s “banging scheme” and use of replay review to electronically steal signs. By the end of the week, Houston’s field manager and general manager had been fired, and the collateral damage from that investigation led to two more managerial dismissals around the league. While the league-administered discipline for non-Houston personnel is still pending, everyone named in the report has been fired.

Now that Houston has been punished, Manfred will turn his attention to the Red Sox, who are under investigation for illegally using the replay room in their own sign-stealing system. Boston’s scheme wasn’t nearly as elaborate as the one used in Houston. Per a report from Ken Rosenthal and Evan Drellich in The Athletic:

Three people who were with the Red Sox during their 108-win 2018 season told The Athletic that during that regular season, at least some players visited the video replay room during games to learn the sign sequence opponents were using. The replay room is just steps from the home dugout at Fenway Park, through the same doors that lead to the batting cage. Every team’s replay staff travels to road games, making the system viable in other parks as well… The Red Sox’s system was possible only when a runner was on second base, or sometimes even on first base. Nonetheless, a team that is able to discern that information live, during a game, and relay it to base runners has a distinct advantage. A runner at second base can stare in at a flurry of catcher’s signs and know which one matters, then inform the hitter accordingly.

Rather than a using a trash can, the Red Sox decoded signs sequences in the video replay room and conveyed that information to the dugout. Once everyone knew the sign sequence, any runner on second base could communicate the signs to the batter via subtle movements or gestures.

The Red Sox will almost certainly face some sort of discipline, especially since they’re repeat offenders: In September 2017, both the Red Sox and Yankees received what now looks like a slap on the wrist for using electronic means to steal signs — on that occasion, using smart watches to communicate between members of the coaching staff and club personnel. Alex Cora obviously wasn’t part of the organization the first time the Red Sox were caught, but he was likely involved this time around. Cora and the Red Sox have already “mutually agreed to part ways,” but that won’t stop MLB from handing down some sort of suspension on top of the penalties it levies on the team.

From an on-field perspective, the limitations of the Red Sox system are immediately apparent. The system only works if there’s a runner on base to see the signs the catchers puts down. The threat of runners stealing signs from second base has always been a part of game — it’s the reason a different sign sequence is used once a runner reaches base — but decoding the sequence using video replay cuts out that gamesmanship.

As I was with the Astros, I was interested in seeing if we could decipher just how much the Red Sox benefited from their own sign-stealing system.

Back in November, I estimated the Astros cumulatively gained around five wins from their banging scheme. Would Boston’s less sophisticated system result in a similar cumulative benefit or would the constraints of requiring a runner to be on second base limit the net effect? The answer might seem obvious, but based on the reports, the Red Sox were able to use their system both at home and on the road, while the Astros were limited to using their cameras at home. That alone increases the sample size for the Red Sox to about half of the total pitches the Astros saw at home in 2017.

A simple look at the Red Sox’s wOBA with a runner on second reveals a big jump in performance in 2018:

Red Sox wOBA with Runner on Second
Year Red Sox League Avg
2015 0.317 0.313
2016 0.348 0.316
2017 0.328 0.321
2018 0.365 0.315
2019 0.349 0.324
SOURCE: Baseball Savant

At the surface-level, something seems amiss. The team’s improvement probably isn’t related to an upgraded roster, as it could have been with the Astros in 2017. Of the 11 batters who accumulated more than 200 plate appearances with the 2017 Red Sox, eight were back in 2018.

When we look at Boston’s plate discipline metrics with a runner on second, things look a bit murky:

Red Sox plate discipline with Runner on Second
Year O-Swing% Z-Contact% SwStr%
2015 30.2% 84.2% 10.3%
2016 27.1% 87.4% 8.7%
2017 27.1% 85.0% 10.1%
2018 25.9% 84.7% 10.0%
2019 29.4% 81.2% 12.1%
SOURCE: Baseball Savant

As a team, with a man on second, the Red Sox chased pitches out of the zone far less often than in years prior but that didn’t necessarily translate to fewer swings and misses. And their contact rate on pitches in the zone actually fell from 2017 to 2018.

With that high-level look not very conclusive, let’s dive into the pitch-level data. As a refresher, I’m calculating run values for every pitch thrown using RE288 — the run expectancy based on the 24 base-out states and the 12 plate count states. When we filter and aggregate those run values, we can get a sense of how a team performed in particular situations, say, when a runner is on second base and they might be relaying the incoming pitch to the batter. To account for the different sample sizes, I scaled the run values to standardize the values per 100 pitches.

Red Sox Pitch Type Run Values with Runner on Second
Year Fastball Breaking Offspeed
Swing Runs
2017 – Runner on 2B -1.43 0.18 -2.71
2018 – Runner on 2B -0.44 -0.67 1.06
Change 0.99 -0.85 3.77
Take Runs
2017 – Runner on 2B 0.83 1.74 2.13
2018 – Runner on 2B 1.06 1.63 2.53
Change 0.23 -0.11 0.40
SOURCE: Baseball Savant

In 2018, the Red Sox saw a huge boost in performance when facing offspeed pitches with a runner on second base when compared to 2017, particularly when they swung at those pitches. They also saw a big improvement when swinging at fastballs. They saw smaller benefits when taking those two pitch types. It’s interesting that they didn’t see any improvement against breaking balls, though their improvement against the two other pitch types more than compensated.

What does the data show when we compare their 2018 performance with runners on second to their performance when there were runners on first or third but not on second?

Red Sox Pitch Type Run Values with Runner on Second
Base state Fastball Breaking Offspeed
Swing Runs
2018 – Runner on, not on 2B -0.78 -1.70 -1.60
2018 – Runner on 2B -0.44 -0.67 1.06
Difference 0.34 1.03 2.66
Take Runs
2018 – Runner on, not on 2B 1.40 1.35 2.12
2018 – Runner on 2B 1.06 1.63 2.53
Difference -0.34 0.28 0.41
SOURCE: Baseball Savant

Almost across the board, the Red Sox performed much better when there was a runner on second. And the biggest difference in performance was swinging at offspeed pitches like we saw above. In Rosenthal and Drellich’s initial report about the Red Sox sign-stealing scheme, there was some speculation that the signs could be stolen by a runner on first base. If that’s true, it doesn’t necessarily show up in the data. They clearly saw a benefit when there was specifically a runner on second.

Finally, let’s add their performance when the bases are empty to our analysis.

Red Sox Pitch Type Run Values with Runner on Second
Base state Fastball Breaking Offspeed
Swing Runs
2018 – Runner on, not on 2B -0.78 -1.70 -1.60
2018 – Runner on 2B -0.44 -0.67 1.06
2018 – Bases Empty -0.68 -0.71 -1.03
Take Runs
2018 – Runner on, not on 2B 1.40 1.35 2.12
2018 – Runner on 2B 1.06 1.63 2.53
2018 – Bases Empty 0.93 0.67 1.33
SOURCE: Baseball Savant

It’s clear that the Red Sox hit much better when there was a runner on second base. They simply crushed offspeed pitches and saw smaller benefits when facing breaking balls and fastballs. The obvious implication is that the team gained an advantage when a runner on second was able to relay the incoming pitch to the batter. When we compare their aggregate run values from 2017 to 2018, I estimate they gained a total cumulative value around five wins — the same estimated benefit the Astros saw with their sign-stealing scheme in 2017. And of course, that doesn’t completely account for the effects that are difficult to quantify, such as the cumulative benefit of requiring opposing pitchers to throw more pitches as hitters lay off of offspeed deliveries designed to entice a swing and a miss.

This isn’t the space to analyze how individual batters benefited from the system, particularly since no players were directly named in the report from Rosenthal and Drellich. I’d expect we’d see similar results to the Astros data though — marginal benefits for most individuals that add up to significant gains for the team. I was surprised to find that the total cumulative effect for Boston was on par with what I estimated for the Astros. It seemed like the limitations of Boston’s system would have resulted in a smaller effect. Perhaps the familiarity of receiving intel from a runner on second allowed Red Sox batters to effectively act on that information when presented; maybe it was hard for hitters to process information from a clanging trash can in the short time before they had to face an incoming pitch.

It appears that both sign-stealing ploys were similarly effective, which makes it’s clear that MLB needs to find a way to prevent teams from implementing these kind of schemes moving forward. Houston’s harsh punishment and the pending discipline for the Red Sox is one step, but the league needs to take more effective preventative measures. Further regulating who is allowed to use the replay room seems like a quick and simple fix that could be instituted as soon as this season. But as long as teams are looking for any and every small advantage and win-at-all-costs attitudes are promoted within organizational cultures, another controversy of this magnitude seems likely to happen again. Rather than react to that new disruption, MLB needs to be forward thinking, particularly as new technology affects how the game is viewed and played.

We hoped you liked reading How Much Did the Red Sox Benefit from Their Sign-Stealing Scheme? by Jake Mailhot!

Please support FanGraphs by becoming a member. We publish thousands of articles a year, host multiple podcasts, and have an ever growing database of baseball stats.

FanGraphs does not have a paywall. With your membership, we can continue to offer the content you've come to rely on and add to our unique baseball coverage.

Support FanGraphs




Jake Mailhot is a contributor to FanGraphs. A long-suffering Mariners fan, he also writes about them for Lookout Landing. Follow him on Twitter @jakemailhot.

newest oldest most voted
ltbailey
Member
ltbailey

We need video replay in MLB!
I know, let’s do that challenge flag system, like the NFL?
So make team’s set up replay rooms where people stare at live game footage for hours on end and have close proximity and communication to the manager on the field?
Yes, perfect!

Nuke it and just get every call right (or maybe wrong, but at least faster!)

HappyFunBall
Member
Member
HappyFunBall

The better complaint is why is MLB using hand signals as if it were still 1910?

mgwalker
Member
Member
mgwalker

Because nothing more complicated is necessary, at least when the opponent is playing by the rules?

(sorry, just never been a fan of more technology being the solution to problems caused by technology)

tung_twista
Member
tung_twista

When playing by the rules already involves needing to change the system depending on where the runner is, it seems unnecessary.
Sign stealing is more of a bug than a feature in my opinion.

mgwalker
Member
Member
mgwalker

All kinds of things in baseball need to change depending on where the runner is — that’s pretty much the essence of the game. A sightline for the runner on second base is the kind of ‘accident’ that adds complexity naturally, which to my mind is the hallmark of a well-designed game.

I don’t think I’ll ever understand the desire to replace baseball’s organic spycraft with headsets and microphones. Where’s the fun in that?

mgwalker
Member
Member
mgwalker

Kinda silly, but I’ll just add: as a player in a local old-man baseball (NOT softball) league, I attest that the man-on-second situation adds a different interesting element:

Almost all catchers are too lazy to put down a sequence, so in principle it would be easy to relay signs. But doing so would definitely be a prick move, given that such laziness in a 45-year-old man squatting behind the plate for three hours is totally justified. So there’s not only the potential for legal spycraft, but also an etiquette that prohibits engaging. This equilibrium becomes unstable toward playoff time…

What a game!

RonnieDobbs
Member
RonnieDobbs

My favorite element of sign stealing is when you get it wrong and the hitter just has no chance… or digging in to hit that FB you know is coming… which you realize is headed at your face. I just realized that you are the other comment I liked. I think we are friends. All these flaws that need to be fixed are what make this game great. Its the fake news style clickbait that makes everything seem broken.

RonnieDobbs
Member
RonnieDobbs

We have got ourselves a baseball fan here.

RonnieDobbs
Member
RonnieDobbs

This is just a fundamental lack of understanding of the game. Stealing signs and the preventative countermeasures have always been part of the game. The problem is the technology – that is the problem which you are confusing for sign stealing.

baubo
Member
baubo

When has human history in its entirety ever suggest that we as a species believe in playing by the rules? Even if you just narrow it to baseball history in its entirety, when had baseball players believe in playing by the rules?

If you think this is just a random few bad apples issue…

https://sabr.org/research/durocher-spymaster-how-much-did-giants-prosper-cheating-1951-finalist

And this is stuff people later talked about. And from the way many players came out against Mike Fiers in the past few months, it’s pretty clear the stuff we don’t know far outweighs the stuff we do know.

mgwalker
Member
Member
mgwalker

Sure, there are always going to be bad actors who bend or even violate the rules. My point is merely that I find it hard, and perhaps even foolish, to welcome more technology as the remedy to its own amplification of this problem.

RonnieDobbs
Member
RonnieDobbs

You want them communicating on little spy wristwatches? There are a lot of reasons that is not a real idea. We all know that digital information is secure!