What Makes a Team Good at Replay Challenges?

We’re somewhere between 25% and 33% into the 2014 season, which means we’re about 25% to 33% into the first season of the Instant Replay Era. Stylistically, I didn’t need to capitalize that, but it makes it feel like a bigger deal. In 2024, it won’t be a big deal, but instant replay is still a shiny new toy that we’re playing and tinkering with, so it seems like a more important thing at the moment. There are a bunch of new rules, new strategies, and new things for managers to think about. So, to paraphrase Ed Koch, how we doing?

Jeff Sullivan, in his handsome wisdom, already talked about replay’s affect on the time of the game. The gist — games are a little longer than last year, but games have been getting longer for some time, so it’s hard to say definitively how much replay has affected that. At the time of this writing, there have been 773 games played. According to Baseball Savant’s replay database (point of order — this is my informational source going the rest of the way), there have been 389 challenges, 60 of which have been issued by umpires. That comes out to just about a challenge per every two games. Is that a lot? We don’t know! May isn’t quite over, but there isn’t much difference in challenges between this month and April. So far, managers have found their stride when it comes to challenging calls.

A long-running debate in our own dweeb corner has been over the affect managers actually have on the team. There are things we can measure like lineup construction, bullpen usage, and proclivity for bunting (the act, not the decoration). There are things we can’t accurately measure like leadership qualities and personnel handling. With replay, one more thing we can measure is presented to us. Exactly what we do with that information isn’t entirely clear, however.

When I’m perusing Twitter in the evening, I will sometimes catch a glimpse of a beat writer call attention to a challenge, how it was ruled, and what the manager’s “record” for challenges is. This is a statistic, and statistics have their own varying degree of usefulness. When a manager has a certain record on challenges, we have a natural tendency to think that a high success rate is good, a poor one is bad. Everyone wants to be successful. There’s whole sections of bookstores devoted to being successful. But it’s the difference between success and failure — in this hyper-specific case, at least — that interests me.

 

Team	Challenges	Over Turned	Not Over Turned	Percentage Over Turned
Marlins	12	10	2	83.33%
Yankees	11	9	2	81.82%
Braves	11	8	3	72.73%
Padres	7	5	2	71.43%
Giants	12	8	4	66.67%
Orioles	6	4	2	66.67%
Astros	11	7	4	63.64%
Royals	10	6	4	60.00%
Dodgers	10	6	4	60.00%
Rangers	12	7	5	58.33%
Rockies	12	7	5	58.33%
Tigers	14	8	6	57.14%
Pirates	16	9	7	56.25%
Brewers	9	5	4	55.56%
Phillies	11	6	5	54.55%
Cubs	19	10	9	52.63%
Nationals	10	5	5	50.00%
Mariners	10	5	5	50.00%
Angels	13	6	7	46.15%
Red Sox	7	3	4	42.86%
Athletics	12	5	7	41.67%
Mets	8	3	5	37.50%
Blue Jays	19	7	12	36.84%
Indians	11	4	7	36.36%
Rays	18	6	12	33.33%
Twins	9	3	6	33.33%
D-backs	6	2	4	33.33%
Cardinals	6	2	4	33.33%
White Sox	10	3	7	30.00%
Reds	7	1	6	14.29%

This is — again, as of this writing — the numbers as far as challenges per team, and the success rate thereof. The Cubs and Blue Jays challenged the most, with moderate success. The Orioles challenged the least, with better success. The Marlins are the best as far as success rate goes. The Reds don’t challenge very often, which is good because they stink at it.

That’s actually not being fair. It’s not as simple as saying Mike Redmond is smart and Bryan Price isn’t. A manager might have different motives for challenging. Perhaps he genuinely thought a call was bad, but maybe he’s stalling to get a guy a few more throws in the bullpen. Perhaps he’s sticking up for one of his players who thought he got jobbed, or maybe he just wants to slow down the game a tad. I can’t say that I’ve seen a suspicious replay challenge, but I haven’t watched every baseball game, either. Winning a challenge might not be the ultimate goal. And even if it is, it’s unclear just how beneficial winning is, anyway.

It’s probably safe to say that being successful at getting calls overturned is a good thing. If your team is on the bad end of a call, getting it reversed helps them. An out is taken away in your favor or added in your favor, and your chance at winning the game is increased at least a little bit. A lost challenge? Well that’s a little harder to figure out.

As it stands right now, the only penalty for a team that loses a challenge is that they get no more challenges. Umpires can still issue one on the team’s behalf after the seventh inning. If a challenge is used on an obvious good call, that would seem to be a bad thing. However, there is no guarantee that another challengeable call is coming, so the point might be moot.

And, of course, not all reversed calls are created equal. On a run expectancy/win expectancy level, an overturned call to keep a runner at first with two outs in the second of a tie game isn’t worth as much as an overturned call to keep a runner at third with one out in eighth inning of a tie game. We can chide the teams who don’t get many calls overturned, but perhaps the successful challenges gave them big chances to win.

In the NFL, a coach that loses a challenge loses another shot at a challenge AND a timeout. This is discernable thing that gets taken away. Timeouts in football are important, especially in the second half. In baseball, the cost of losing a challenge is much less. No outs are taken away. No limit on pitching changes are invoked. If you try something and it doesn’t work, the worst thing that happens is that you can’t do that thing anymore. If it does work, there’s a better chance that good things will come to fruition. This is the basic idea behind dating culture as well, though that doesn’t seem to be relevant here.

As baseball thinkers, we look at success rates as a fairly binary thing. Stolen bases are good, getting caught stealing is bad, and it’s important to do the former more than the latter. Things like OPS and ISO are success rates in their own way. But with challenges, the gains so far outweigh the cost, that perhaps success rate isn’t as important. Perhaps the Blue Jays, Cubs, Rays, and Pirates have the right idea. If it’s close, challenge it. The WORST thing that could happen is the rest of the game would have to be managed as if it were 2013. Heck, the umpires can still bail you out after the seventh inning if they think they really borked a call.

Come on, Orioles! Time to get more challenge-happy! Hey Reds, don’t let your poor showing so far keep you down. You got pretty much nothing to lose! YOLO, etc.!

Twenty years from now, someone will have done a study that looks at challenges and leverage indices and win probabilities and we’ll be as sure about challenging as we are about bunting (still not the decoration). Until then, it seems to make sense to keep throwing pasta at the wall. If it sticks, good for you. If it falls, so be it. It’s not like you need to ice the other guy’s kicker or anything.