Robot Umps and Velocity Incentives
Had a few calls gone differently in the ninth inning Sunday night, everyone could have gone to bed earlier. The poor beat scribes could have returned home, or to their Marriott Courtyards or SpringHill Suites, at a reasonable hour. I wouldn’t have fallen asleep on my couch. And the Cubs would have scored victory of sorts as they would have arrived to their charter flight earlier. The Cubs traveled to Denver after the game. The first night in adjusting to altitude in Denver after an 18-inning affair in Chicago is the kind of game some believe you can pencil in for a loss.
If Austin Romine could have had a better night receiving in the ninth inning, everyone could have enjoyed an earlier night. Romine is a league-average framer. Of course, it’s not easy to catch Aroldis Chapman.
Others have studied velocity and the affects it has on framing, including FanGraphs’ Jeff Sullivan way, way back in in 2013 when framing was still a relatively new development. Sullivan devised his own home-made recipe to quantify the effect:
For starters, the correlation between average fastball velocity and Diff/1000 is -0.26. For relievers, the correlation between average fastball velocity and Diff/1000 is -0.34. These aren’t real strong, but they are meaningful, and they don’t account for catcher identity. What the numbers show is that, the harder a guy throws, the less favorable a strike zone he gets.
For the first time, this season, we, the people, have the ability to search a more detailed strike zone at BaseballSavant.com. Instead of 13 zone to study, there are 29. I was curious to look at that new gray area on the edge of the zone you can search at Savant, zones No. 11-19 on the detailed search. I wanted to study borderline calls there by velocity. What I found:
Fastballs 92 mph or less: 2,805 called strikes, 2,602 balls (51.8% called strikes)
Fastballs between 93-96 mph: 2,474 called strikes, 2,627 balls (48.5% called strikes)
Fastballs 97 mph or greater: 251 called strikes, 363 balls (40.8% called strikes)
So a league average fastball, this season, has been called a strike 11% more often than a high-velocity fastball on the edge of the zone. That’s interesting.
Chapman threw eight fastballs in the border region Sunday night, five were called for balls. While many hitters probably feel Chapman has plenty advantages as is he could benefit from another one: a robot ump.
The balls:
The strikes:
This is not one of the pitches studied, but consider the amount of momentum created by this pitch to have Romine nearly skid into the home team’s on-deck circle.
This is also not one of the pitches in question, but it is glaring evidence to demonstrate it is not easy to simply to catch a 100 mph fastball if it is off target.
Catching Chapman ain’t easy.
The bottom of the ninth inning began Sunday night, with Chapman walking Addison Russell without controversy. Chapman missed badly with location.
Jon Jay followed by singling on a 2-2 pitch, but consider the 1-2 offering:
The 1-2 pitch was a ball, just missing the corner. But consider the movement of Romine, moving not just his glove but his upper body moving to his left, perhaps needing to get his body behind the ball to limit his glove being demonstratively moved out of the zone.
After a Wilson Contreras strikeout, Chapman began Albert Almora with this pitch, which was unjustly deemed a ball. We know the difference between a 2-1 and 1-2 count is nearly 200 points in batting average. So that missed call represents a significant swing in probability. Again, the 99 mph pitch caused some recoil to Romine’s glove.
Almora singled in Russell to cut the Yankees’ lead to 4-2. Javier Baez followed with a single to score Jay and make it a 4-3 score with one out. Chapman struck out Kyle Schwarber for the second out and then faced the reigning NL MVP with. The one-strike, one-ball offering to Bryant:
If Chapman gets that pitch perhaps he more aggressive attacks Bryant, gets the third out, and the Yankees win. Instead, Bryant is ultimately intentionally walked to load the bases and Chapman hits Rizzo to tie the score at 4. Nine innings later, the game ended.
Chapman did not get many benefits of the doubt in the ninth inning, rare velocity doesn’t help a pitcher in every regard. Sure, you’d rather have elite velocity than not have it. But sometimes it can hurt an arm.
Sunday was a glaring reminder that velocity must be taken into account when evaluating a catcher’s framing, along with playing on the road and trying to close out a game in the ninth inning in a hostile environment. Umpire ball-strike bias explains much of home-field advantage. Sunday night was also a reminder that as great as Chapman’s stuff is, sometimes it can work against him.
It is Chapman, and the other high-velocity arms in the game, that would benefit most from an automated zone. An automated zone, would make some of the most dominant arms even better.
A Cleveland native, FanGraphs writer Travis Sawchik is the author of the New York Times bestselling book, Big Data Baseball. He also contributes to The Athletic Cleveland, and has written for the Pittsburgh Tribune-Review, among other outlets. Follow him on Twitter @Travis_Sawchik.
This is very interesting. Any time a significant change is made, it carries effects that are not immediately obvious.
Lucky for us, you’re here to delve into these effects!
I obviously missed the end of the game, because an earlier version of this post had the Yankees losing. It’s since been fixed. My apologies for the error.
I may have missed it, but I keep hoping to see an article attempt to quantify how much shorter games would be with a robot ump if at all. Human umps tend to call more strikes in hitters counts more frequently and more balls in pitchers counts. It would seem like it would force action (which may lengthen games in a good way) or make 1-2-3 innings quicker.
Plus no arguments over balls and strikes!
Beyond even the umpire bias, games would certainly go quicker with an automated strike zone. Think of all the framing and holding of pitches that goes on in a game, on a high percentage of pitches. Major League catchers are taught to stay still, limit movement, and “present” the pitches to get the calls. Sure, each instance is small in itself, but the holding and presenting of so many pitches certainly drags the game out longer when added up over 9 (or 18) innings. Imagine if catchers could just catch and return the ball on every pitch with no regard for the effect on the ball/strike call.
I feel like walks would increase sharply. Many 3-0 pitches are not strikes, but get pity calls anyway.
I am so conflicted about the idea of robot umps or perfect strike zones. On the one hand, I want them to get the call right, just like with replays in the field. Getting the call right and following the rules of the game are hugely important. But on the other hand, framing is a real skill and a really cool one, and I think I would miss it if it were gone. It’s part of the defense.
And home-field advantage would also be eroded … I am also conflicted
It doesn’t have to be that way. All the unemployed umps can hire themselves out to pull 4AM fire alarms in the visiting team’s hotel!
I’m aware you’re joking, but there aren’t any unemployed ump caused by making the change to robo-umps. Was the ball a foul tip? Hit by pitch (yes/no/yes but in the zone and thus a strike/yes but no attempt to evade and thus a ball) is still a home-plate umps call.
There’s still plenty for the home plate ump to do even with robo-ump.
HappyFunBall, living up to the name.
Eroded, but isn’t there something cool about the advantage being based exclusively on how players and stadiums match up? The effect of fans/stadiums on umpire and player comfort is a factor in nearly every sport. But baseball is one of the only sports where you can construct a major advantage without crowd effects. It’d be kind of neat to emphasize that aspect even more.
I don’t care how hard a guy is throwing, that 1-1 pitch to Kris Bryant was an obvious strike. Especially considering Lester was getting about 1-2 inches off the plate called for strikes the whole damn game. Too bad the umpire’s union is apparently too strong for MLB to do anything about it.
Ah yes, only the team you are not rooting for ever gets calls outside. Your team always gets the shaft, no one else. Life is rough bro
I was illustrating a point that West had established the outside corner as a strike the entire game and apparently went back on it on one of the most important pitches of the game. Re-watch the game if you don’t believe me.
He was awful for both sides, and he is an awful ump in general, don’t jump to conclusions.
This is an implicit bias we all have — “Headwinds/Tailwinds Asymmetry” as pointy-heads call it. http://freakonomics.com/podcast/why-is-my-life-so-hard/ Important to recognize even when it’s only baseball.
Also important to recognize when it isn’t happening, like in this situation.
This bias you mention is only about how fans remember poor calls against their teams. Every team has poor calls go against it. Bias does not change the fact that the calls were in fact bad and effected a game’s outcome.
The 1-1 to Bryant is absolutely Romine’s fault. If he just stuck his glove where he caught it, I bet they get the strike. Jabbing his glove down and in tells the umpire that Romine didn’t think it was a strike.
If the strike gets automated, would catchers stop crouching altogether, since there would be no reason to have to frame pitches well?
Probably not. The glove provides a visual target for the pitcher to hit, which increases accuracy.
No, it’s easier to protect yourself when crouched, your probability of stopping a wild pitch are much higher when crouched, and it’s also probably healthier and more comfortable long term, as opposed to standing bent over.
With nobody on base, why worry about wild pitches? I would think catchers could still leave a sufficient target, and maybe step farther back, or move to the side to avoid foul tips. Maybe they wouldn’t stop crouching all the time, but I’m tempted to think there are a few times it would be of benefit.
I was watching this game live. Joe West was missing calls on for both teams throughout the game. Robot umps are not the issue. Joe West’s strike zone was not consistent at any point. Brian Duensing had a few pitches for strikes called balls, and he definitely is not a man of elite velocity.
I do not disagree with the overall data that league average velocity will net a player a borderline strike 11% of the time, but does that number stay the same when you take Joe West out of the equation?
Does seem like “robot ump” and “Joe West” appear together with shocking frequency in articles/comments, no?
Add “Vic Carapazza” and “Angel Hernandez” to that group too.
Chapman was missing his targets pretty badly on some of those pitches. Even if they should have been strikes, hard to fault Romine for not framing well when he has to move his glove 3 feet diagonally quick enough to stop 100mph.
Every single baseball player (minus elite 1 trick framers) would benefit from robo umps. At least a guy would actually know what his strike zone is. Also every single fan would benefit more from robo umps too. If they want to speed up the game and showcase their stars, then stop letting umps be the show and dictate the drama. Let’s let the game play itself out. I hate umps. I hate them mostly for being human. These mistakes cost teams and players millions. The technology is there. I have to wear a seatbelt because law, baseball should have to finally get ALL the calls right. Its been too long.
We’d still need umps for the basic out/safe calls, and to determine foul tips or HBP (without going to a review every time). So it’s not like umps lose their jobs.
It’s true, you lose the “art” of pitch framing. But 5-10 years ago nobody really knew what “pitch framing” was about, and Catchers were still very valuable for their calling and throwing skills. I think I’d be fine with moving to robot umps, and within a few years we’d probably wonder why we ever had umps call balls and strikes since having the tech to do it automatically.
Calls on the bases, sure, but I bet the RoboUmp3000 would be fully capable of determining fouls and HBP based on a sudden change in the direction of the pitched ball. In fact, the RoboUmp3000 may even be better at that too.
Greetings. I am the AWESOM-Ump 4000. I have been sent from MLB to serve as your personal umpire.
I very much agree that umps will always be needed on the field, for the reasons you mention. I’m just terribly sick of bad calls. So often a team’s whole season can come down to a called strike here or there. I just want them to be called correctly. I want this so badly.
I recall being a young sports fan. Maybe I was 10 or 11. I got really into football at first even though I played baseball much more. I remember being so put off by MLB because of the inconsistency of their umpires. Sure every league’s officials make mistakes all the time, BUT ONLY BASEBALL forces it’s officials to make hundreds and hundreds of calls per game.
Surely a statistically passionate web community such as FG can easily grasp how if given soooo many opportunities for human error, you eventually reach a point where human error is all but guaranteed to effect a game’s outcome.
Edit; Also framing is just a fancy word for cheating imo. Sure it’s not breaking any rules but a good framer is simply good at deceiving the umpire =/= cheating.
“…human error is all but guaranteed to effect a game’s outcome.”
Not true. If the error is close enough to random it simply reduces the distance between teams’ expected performance. In statistics it’s called nondifferential misclassification and it’s part of why even the worst teams win 30-40% of the time.
You’re correct but in the SSS of a single game that randomness can easily be tilted one way or the other.
And I would argue that a fairly even distribution of talent is why even bad teams still win 30-40% of their games. Even bad teams have good players and sometimes good teams play poorly.
The result of a contest between a bunch of humans will never be completely predictable. But we can at least get all the calls right and let what actually happens decide the game instead of 1 human’s interpretation of it.
Also catcher’s interference, although that would occur less if catcher’s weren’t reaching forward to catch and frame the pitch, batter’s interference on steal plays, and check swings.
With less to focus on, I wonder if umps would actually call batters for failing to move out of the way of a pitch and not award as many bean balls.
It’s also worth mentioning how frustrating it is as a fan to see your one of your favorite players get ejected for arguing balls/strikes. If I paid to go that game I’m immediately pissed off. If I’m not a diehard fan and I paid to see Miggy, but he hot ejected in the 1st, now I’m probably a lot less likely to spend money in the future on a live game.
I don’t know if you meant that people haven’t been able to quantify the value of pitch framing save for the last 5-10 years, but I think a catcher’s receiving ability is something that people in baseball have valued for quite a long time.
‘get ALL the calls right’. Do robo-zone advocates realize that the baseball rulebook does not even define ‘strike’ to the level of precision at which statcast can measure pitch trajectory? That is, the rulebook does not specify whether the pitch must remain over the plate from front to back, or whether it is acceptable to knick just a corner and then leave the zone before crossing the plane that is perpendicular to the back of the plate.
Not only would robo-zone make the game less interesting by removing the pitcher/umpire batter/umpire catcher/umpire dynamics (which develop over innings, games, seasons and careers), but it would require some programmer to re-define what in fact constitutes a strike.
There is absolutely nothing ‘unfair’ about leaving balls/strikes to the judgement of an impartial human official, even if that official makes mistakes some fraction of the time (as long as those mistakes are random). One important and unrecognized ‘human’ factor here is the tendency to remember only the bad calls that go against your team.
PS: As a former high-school catcher, I have a deep appreciation for the craft of pitch framing. Regardless of when it became possible to quantify this skill, the fact that it is a skill has been understood by catchers since the very beginning.
I don’t believe any human can be impartial. Especially with all the “dynamics” in play you mentioned. My team team shouldn’t lose because the ump liked the other team’s catcher’s cologne better.
I’ll wager that your team has never lost because the umpire preferred the other catcher’s cologne. His superior ability to frame pitches, probably. And that – reward for a well-developed skill – is exactly as it should be.
Well we’ll have to agree to disagree. I don’t feel Bryce or Trout or Miggy should be given a bad call against them because of a “reward for well developed skill”. Personally I’d rather see Trout, Bryce, and Miggy be rewarded a ball for their superior eye.
Also the cologne comment was tounge in cheek. Basically I’m inferring that some umps simply like some catchers more than others thus those catchers likely recieve more favorable calls more often which I feel is BS.
Ok I agree to agree to disagree. But I’ll bet that robozone only works against Trout/Cabrera, who would no longer benefit from the “If you throw a strike, Mr. Williams will let you know” effect.
I honestly doubt that. If Votto’s strike zone never ever changed at all do you seriously think he would do more poorly?
mgwalker, I’ll take that bet, not that we’d ever be able to prove it one way or the other. Humans are subconsciously affected by all kinds of external factors, and smell is an important one. (That’s why when we’re suspicious of something but can’t say exactly why, we say it “doesn’t smell right.”) A pleasing odor can put an umpire in a more positive, accepting and generous mood, whereas an unpleasant odor can equally put him in a more negative, mean-spirited frame of mind. (If you don’t believe this, google “pheromones”.) Many games turn on one or two close ball/strike calls, and out of the many thousands of major league games that have been played over the years, I’ll bet that there have been at least a few where the odor emanating from the catcher had enough of a subconscious influence on those calls to tip the balance one way or the other.
As for framing, in my opinion the practice of deceiving umpires into making incorrect calls is not something that should be encouraged or rewarded.
Nah. The rulebook says you can nick the back corner with a backdoor strike.
It defines a strike as “any part of the ball passes through any part of the strike zone”, and the strike zone as a three-dimensional solid over home plate.
“An automated zone, would make some of the most dominant arms even better.”
Wouldn’t the opposite happen? I would think a fixed strike zone would benefit batters overall. More players would have a “feel for the strike zone” if it was consistent. Pitchers wouldn’t be able to move a batter’s eye level or feet as easily, if the batter always knew their starting position could be the same every time in every park. There also wouldn’t be the distraction of trying to figure what qwerk the umpire has that day.
It would remove variance. Hitters with a good eye would be rewarded in that their own good judgement would not be confounded by the umpire’s. Pitchers with better command would be rewarded for being better able to pitch to the edges of the zone. Those on both ends of the contest with poorer skills would be hurt because they would no longer be able to rely on luck, framing, or influencing the umpire.
Agree this is the direction it would take. You’d have clearer separations between good vs bad players at each skill.
Nobody can rely on ‘luck’ for more than a few games. Reliance on framing, and thereby influencing an umpire, is an example of a good skill.
I think it would work both ways. Pitchers that are good at nibbling the corners would probably be helped a lot as well as hitters with exceptional eyes. It would make the talent shine brighter I honestly feel.
I’d bet that the home-team 9th inning bias completely overwhelms the hard thrower bias. Chapman’s home and away strike zones would, I’ll venture, look like they belong to two different pitchers.
Recent work presented at SABR Analytics showed that umpire strike zones depend not only on location, velocity, platoon configuration, and count but also on pitch vertical and horizontal movement. More at
http://vixra.org/pdf/1703.0163v1.pdf
An automated zone would also tend to give us more strikeouts, more walks, and fewer balls in play, which I’m not sure I want to watch.
Human umps call a small zone in pitchers’ counts, and a large zone in hitters’ counts. This might be human psychology, or might be rational given limited human vision — reflecting the fact that zone rate really is lower at 0-2 than at 3-0. But robots with lasers won’t have human limitations, they can be true to the pitch.
So for the same pitch locations, we’ll get more strike calls on 0-2 and more ball calls on 3-0. Pitchers will learn the new zone and claw back some of the increase in BB + K, but probably not all of it.
(Hi, I didn’t read all the comments.)
The robot umps can show the same count biases as human umps. Though I have no idea what governance process would result in this outcome.
Oh we could make them, but then they’d be deliberately making calls that are wrong by the rulebook. Will anybody go for that? I think then the traditionalists are mad and the accuracy mavens are mad.
Maybe, or maybe we see more hits on 3-0 cause a pitcher knows that same pitch he threw 0-0 for a ball is still gonna be a ball so instead he pipes one down the middle because he has to.
There are more pitches out of the zone called strikes than pitches in the zone called balls, so strikeouts would go down and walks would go up. I believe BIP would stay about the same, as the strikeout decrease is approximately equal to the walk increase.
there’s enough technology already to evaluate the precision and accuracy of ump calls. just fire people who suck after some defined period of time…seems that’s the real issue. robot umps is a video game; there’s enough of that already. random human variation is an aspect of life. it isn’t always binary. it kinda makes it fun.
I noticed that I’m disappointed that I can’t get calls in video games for painting the corners like you’d expect to in real life. It’s tough to take the human element out.
I’m curious if we would get “trick shot” pitchers with a robot ump because the zone would be so clearly defined. Can you learn how to barely catch the zone and exploit the robot’s defined zone? What happens when a guy decides to bounce the ball over the plate like in cricket? Bounce it high enough, it’s a strike?
If a pitcher is “exploiting the robot’s defined zone” as you describe, then he is throwing legal strikes that should be counted as such.
As for pitches that go through the strike zone after bouncing, I suspect the robo-umps could be programmed to account for that; but if not, there will still be a human home-plate umpire there to overrule the robot and call it a ball.
Accountability goes a long way toward enforcing competency. I would be on board with this as an alternative to the robo umps.
Or even stop letting the bad umps call balls and strikes. Let the ones with better eyes for it do it more. Make the bad umps stand by the foul poles or even send em down to the minors. Treat the umps more like the players instead of auto-workers.
“So a league average fastball, this season, has been called a strike 11% more often than a high-velocity fastball on the edge of the zone.”
This is really interesting and concerning, however, I wonder if there is a reason other than “it’s harder to see.”
Hypothesis #1:
Harder throwers are wilder and have a wider distribution of pitch location. Therefore, the harder throwers will have more pitches on the outer edge of the gray area than softer throwers.
Hypothesis #2:
Harder throwers are generally relievers. It takes time for the umpire to adjust to calling pitches from a new pitcher. A reliever doesn’t have the benefit of time and therefore gets fewer strikes called.
Hypothesis #3:
Softer throwers are more control specialists. They work the corners and eventually get the umpires to expand their strike zone.
I think all 3 have merit. It’s probably a combo meal going on there.
I honestly don’t think there was a difference between the strike zone that Chapman got with the other 15 or so pitchers that appeared in the game. Having watched every single pitch, minus one or two for bathroom breaks, Joe West was all over the place all night. There were obvious strikes called for balls and obvious balls called for strikes for both teams, and I am sure every single pitcher has a complaint about the way it was called. I honestly thought around the 6th inning Country Joe just started calling pitches randomly for a while, or it felt like it.
I knew this would be a terribly called game when Joe West was at second base on Friday.
I personally don’t care one way or another for robot umps. What I do care about is getting the umpires that are terrible out of the game and replacing them with capable umps that can do the job competently. At the end of the day, no sport will ever be called perfectly. The goal should be to minimize the errors, and keeping guys like Joe West around just makes that goal impossible. If I did my job as bad as he did, I would be fired, as it should be.
Late to the party, but something I haven’t seen specifically mentioned in regards to implementing robo umps: Would we miss having that extra half inch/inch given to pitchers when they hit their spot? On one hand, it’s incredibly frustrating when it happens to your batter, but on the other, it’s very satisfying when your pitcher is rewarded for his control. Could a system be put in place that would still account for this? If the catcher selects “outside” (either by setting up, or some sort of button?), the robo-ump adds a half inch to the strike zone on that side, while subtracting 3 (or 6?) inches from the other. I don’t think we would ever see that implemented, but it is interesting to think about, as I’m sure the guy who mentioned trying to stretch the zone in the video game would agree. I feel like it would maintain some of the “framing” aspect, but solely based on picher skill and control, rather than catching skills and umpire bias/human limitations.