How Much Did the Astros Really Benefit from Sign-Stealing?

With the offseason moving at it’s now customarily glacial pace, the story that’s dominated the baseball headlines for the past week has been the Astros sign-stealing scandal. Since the first story was published by Ken Rosenthal and Evan Drellich at The Athletic last Tuesday, we’ve seen bits of evidence confirming the Astros strategy: a crude system of live monitors and a trash can to relay the incoming pitch type to the batter. On Friday, Rob Arthur of Baseball Prospectus provided audio evidence from the TV broadcasts, on which the whacking of the trash can can be identified and analyzed. Rosenthal and Drellich followed up their post with another revelation that the Astros were exploring different ways to electronically steal signs. And then there’s the video evidence pulled from the 2017 Astros World Series documentary showing a computer — sometimes a monitor, sometimes a laptop — in the tunnel leading to the dugout with a trash can nearby.

MLB is still conducting its own investigation, but the evidence that’s been dug up publicly is pretty damning. Sign-stealing has always been part of the game, but the Astros crossed the line when they began using electronic means to relay information in real-time. And this isn’t the first time they’ve been accused of baseball skullduggery of some sort. There’s little doubt that some form of punishment will be handed down by the commissioner’s office once MLB wraps up its investigation. The only question is how harsh will the penalties be; will it be a slap on the wrist, or will the hammer come down hard to deter other teams who are likely exploring other ways to use technology to steal signs.

Gamesmanship and various forms of cheating are as much a part of the fabric of baseball as the unwritten rulebook, all in an effort to gain an upper hand against your opponent. But how much of a benefit does sign-stealing give to a batter? Knowing what type of pitch is incoming gives the batter a distinct advantage — more information is far better than just guessing — but they still need to translate that information into action, whether its a swing or a take. Since we have plenty of data to analyze, can we measure the effect of sign-stealing on the outcome of a pitch, play, or game?

In 2016, the Astros’ home strikeout rate was 24.5%, the third-worst in baseball. The next year — the year they supposedly implemented their sign-stealing system — it plummeted to 16.7%, the best in baseball. While it may look related, it’s probably more coincidental. Their strikeout rate away from home also dropped by four and a half points and they’ve managed to keep their overall strikeout rate below 20% over the last three years. Besides, sign-stealing should have far broader effects than simply reducing a team’s strikeout rate. Knowing what type of pitch is coming should also help contact rates and, perhaps, produce more high-quality contact.

Digging into the Astros plate discipline stats helps us get a little more granular. Here’s a comparison of some key metrics at the team level in Minute Maid Park.

Astros Plate Discipline At Home
Year O-Swing% Z-Contact% SwStr%
2016 28.60% 79.15% 13.20%
2017 27.80% 85.85% 9.50%
Change -0.80% 6.70% -3.70%
SOURCE: Baseball Savant

The Astros improved in all three of these plate discipline metrics from 2016 to 2017. They chased pitches out of the zone less often, they made contact on pitches in the zone more frequently, and swung and missed less often. Then again, Houston posted better marks in each of these three metrics on the road than at home in 2017.

The Astros roster had some pretty significant turnover between these two years. Could the difference in talent level explain their huge improvements in plate discipline? Five of the nine batters who accumulated the most plate appearances in 2016 carried over to the next season: George Springer, José Altuve, Carlos Correa, Marwin Gonzalez, and Evan Gattis. If we limit our data query to those five batters, we see similar results.

Astros Plate Discipline At Home
Year O-Swing% Z-Contact% SwStr%
2016 27.91% 81.91% 11.50%
2017 27.45% 86.51% 9.41%
Change -0.46% 4.59% -2.09%
SOURCE: Baseball Savant
Combined plate discipline for George Springer, José Altuve, Carlos Correa, Marwin Gonzalez, and Evan Gattis.

These five batters improved their plate discipline across the board. And when we compare their plate discipline on the road in 2017, they posted a better chase rate and swinging strike rate away from home. Only their zone contact rate was better at home in 2017 (86.5% at home, 85.7% on the road).

At a broad level, it’s hard to make any conclusive statements about the possible effects of the Astros sign-stealing. Travis Sawchik came to a similar conclusion in an article posted at FiveThirtyEight yesterday. But what these plate discipline metrics fail to capture is the true impact on a per-pitch level. The additional information provided by the thumping trash can definitely helps a batter make a decision about whether or not to swing. Would it result in higher quality contact and better takes on borderline pitches?

You’re probably already familiar with RE24 — the run expectancy based on the 24 base-out states. As part of his work on Baseball Savant, Tom Tango recently introduced RE288; that is, the run expectancy based on the base-out state and the plate count state. That gives us an easy way to calculate run values for any pitch thrown and then aggregate those run values to begin to analyze how a team, say, benefited from specific knowledge about the pitch being thrown. Knowing what is coming on the first pitch of an at-bat isn’t nearly as important as knowing what pitch is coming in a 3-2 count with runners on base.

I pulled pitch data from Baseball Savant for 2016 and 2017 and calculated run values for swings and takes against the three major pitch types: fastball, breaking ball, and offspeed. At a team level, here’s what we see:

Astros Pitch Type Run Values At Home
Year Fastball Breaking Offspeed
Swing Runs
2016 -83.52 -61.36 -15.12
2017 -44.15 -24.81 -17.2
Change 39.37 36.55 -2.08
Take Runs
2016 79.96 30.01 23.66
2017 75.23 43.46 21.25
Change -4.73 13.45 -2.41

In 2017, the Astros massively improved their performance against both fastballs and breaking balls when playing at home, particularly when they swung at those pitch types. They also got better at watching breaking balls pass by. Their performance against offspeed pitches didn’t really budge, but I don’t think that tells us anything. Knowing whether a pitch is a fastball or a breaking ball is far more important because of the way those types of pitches move. If a fastball is signaled, the batter can gear up for heat and make sure they put a good swing on a relatively straight pitch. If a breaking ball is signaled, they can focus on the movement of the pitch to remove the deception it would otherwise create. If an offspeed pitch is signaled, the batter still has to recognize the speed of the pitch, battling their eyes and instincts that tell them a pitch that looks like a fastball was thrown.

I also calculated run values for swings and takes on pitches in and out of the zone. When paired with the plate discipline metrics above, their improvement in one single area comes into focus.

Astros Zone Run Values At Home
Year In Zone Out of Zone
Swing Runs
2016 -53.8 -77.82
2017 20.8 -75.08
Change 74.6 2.74
Take Runs
2016 -114.56 214.33
2017 -107.37 214.99
Change 7.19 0.66
SOURCE: Baseball Savant

The Astros improved by nearly 75 runs when swinging at pitches in the zone in their home ballpark. We’d expect to see a big swing in run values given the Astros’ improved zone contact rate discussed above. The other three facets above also show improvements too.

What do these run values look like when we compare their 2017 home performance to their away performance?

Astros’ Pitch Type Run Values
Year Fastball Breaking Offspeed
Swing Runs
2017 – Home -44.15 -24.81 -17.2
2017 – Away 16.08 -7.36 -11.97
Difference -60.23 -17.45 -5.23
Take Runs
2017 – Home 75.23 43.46 21.25
2017 – Away 84.97 25.4 22.1
Difference -9.74 18.06 -0.85
SOURCE: Baseball Savant

Astros’ Zone Run Values
Year In Zone Out of Zone
Swing Runs
2017 – Home 20.8 -75.08
2017 – Away 79.99 -96.45
Difference -59.19 21.37
Take Runs
2017 – Home -107.37 214.99
2017 – Away -83.63 229.33
Difference -23.74 -14.34
SOURCE: Baseball Savant

Well that certainly clouds our data. In almost every respect, the Astros performed better on the road than at home in 2017. The only aspects where they saw better results at home were when taking breaking balls and taking pitches out of the zone. They were around 20 runs better at home than on the road when they spit on a breaking ball or if the pitch was out of the zone. Considering all the data we’ve seen so far, that might be the one area where the Astros truly benefited from their sign-stealing system, and it earned them maybe a couple of extra wins in 2017.

There are so many factors at play, it’s nearly impossible to isolate the effects the Astros’ sign-stealing had on their performance. The fact that they continued to use this scheme throughout the year tells me they found some benefit, even if it’s hard to determine exactly what it was from the data. In the end, the Astros improved across the board in 2017, leading to their first World Series in franchise history. That championship was fueled by new talent, player development, and possibly cheating. It’s likely some combination of the three, and the data is murky enough to leave it at that for now.

Jake Mailhot is a contributor to FanGraphs. A long-suffering Mariners fan, he also writes about them for Lookout Landing. Follow him on Twitter @jakemailhot.

Newest Most Voted
Inline Feedbacks
View all comments
4 years ago

A lot of people bring up the BB/K stat, which is fair, but what’s interesting to it for me is that Brantley, Altuve and Gurriel have all never struck out over 90 times a season, and Bregman has historically been an above-average OBP bat. It’s not to say they’d be the same, but it’s an interesting thought to me with walks/strikeout measurements since this is a squad somewhat known for good batting eyes.

The Guru
4 years ago
Reply to  hombremomento

Gurriel came up in the astros as did bregman…..they have good batting eyes because they know whats coming.

4 years ago
Reply to  The Guru

You can check Gurriel’s stats outside of the MLB. Gurriel has never struck out over 65 times in a season whether it was the MLB, CNS or NPB. Similarly, Bregman has had high OBp and walk rates in every level of ball he’s played in.