Searching for This Year’s Called Balls on Pitches Down the Middle
This is one of those posts I like to write over and over. We’re always getting new information, meaning we’re always getting new borderline calls, and when there’s room for error around the fringes, that means there’s a non-zero chance something could go more dreadfully wrong. Like, say, a pitch being taken down the middle, and getting called a ball. Humans are perfect at nothing, not even the things that we think we’re perfect at, and a baseball season has a whole lot of pitches in it. Lots of opportunities for funny, uncommon mistakes. When it occurs to me, I try to find them.
I don’t do it because I delight in pointing out when umpires mess up. I really don’t, because their job is harder than my job, and I don’t like to pile on. Everyone’s capable of stupid mistakes. I do it because, think about it. We’re used to seeing questionable calls around the edges. But, right down the middle? It’s the kind of mistake you want to investigate, because you feel like something must’ve happened. My goal when I do this is to try to understand why the call got made how it did. Find an explanation for the seemingly inexplicable. I don’t know why this interests me so much, but, here we are, and no one on staff has told me to stop.
We’ve had weeks of baseball in 2015. There’s nothing particularly significant about right now, but let’s reflect anyway on what’s taken place. Let’s search for those called balls on pitches taken down the middle.
As usual, the key resource is Baseball Savant. Baseball Savant to find the data, and MLB.tv to watch the pitches and recover images, once I know where to look. Savant allows you to select pitches by zones and results, so I searched for balls in the middle third of the strike zone, horizontally and vertically. I was left with a total of five candidate pitches — five pitches, in just a few weeks, thrown down the middle and called balls. Below, reviewing those pitches, in chronological order.
1
The brief list begins on April 8, with Brandon McCarthy pitching to Yangervis Solarte in the second inning. The home-plate umpire: Mike DiMuro. The target pitch was an 0-and-1 sinker. The location of the pitch:
Definitely no way around it — that pitch was right down the middle. You can easily see it’s over the middle of the plate, and you can just as easily see it’s at the level of the hitter’s mid-thigh. There were no complicating factors, like runners on base. This is a pitch where it would be totally inexcusable to call a ball. Strikes don’t get any more strike-y. Which is why, in reality, this pitch was called a strike.
PITCHf/x, see, went and got itself confused. It thought the above pitch was a ball, but this was the true 0-and-1 called ball:
Still questionable! You could maybe make an argument for a called strike. But it wouldn’t be a strong argument, and this most certainly wasn’t a called ball on a pitch down the middle. So, one pitch in, we have a data glitch to blame. Let’s move on to the other four.
2
We return to Dodger Stadium, a week later. It’s Danny Farquhar facing Adrian Gonzalez in the eighth inning on April 15. According to the spreadsheet, the mistake came on the first pitch. Here’s Farquhar’s first-pitch fastball, which Gonzalez took:
Problem. The pitch was over the plate, but it wasn’t over the middle of the plate. And more importantly, look at the height of that thing. The sheet says the pitch was two and a half feet off the ground, with the top of the strike zone three and a half feet off the ground, more or less. That pitch was at belt level, with Gonzalez standing almost straight up. That’s not a pitch down the gut. That’s a borderline high strike, but Marvin Hudson called a ball. We have another glitch, and actually, now that I think about it, the whole inning was kind of glitchy that day. We should keep going. No need to dwell on these errors.
3
Let’s stay in California! But head north, to San Francisco. On April 17, Yusmeiro Petit pitched to Chris Owings in the top of the eighth of a developing blowout. The home-plate umpire was Chris Segal. I’m told the first pitch was the poorly-called pitch. Checking it out:
OK, no. Got me again. Not only was this not a called ball down the middle — that very pitch got away from the catcher, and both runners moved up. I’m not sure what PITCHf/x thought it was seeing. No, let me take that back. I am very sure what PITCHf/x thought it was seeing. I just don’t know why it saw it. It’s not just humans who tune out of blowouts. To pitch candidate No. 4!
4
Apparently we can’t get out of California. Now it’s down to Anaheim, for a game on April 21. In another blowout, R.J. Alvarez faced Chris Iannetta in the seventh inning. It’s claimed Marty Foster made a bad call on a 1-and-2 slider. Maybe you can’t blame someone for briefly taking his eye off a nine-run contest. Let’s look at this pitch:
There’s that slider, where we expected. But there’s also that other thing, which was unexpected. Can’t call a ball on a swing. Can’t call a ball on a foul. Can call a foul on a foul. Foster nailed it. I guess “ball” and “foul” sound not completely different, and they both end with an L. But that’s not how PITCHf/x came up with this. There’s just one pitch candidate left.
5
And it takes us to Arizona, on April 28, shortly before Archie Bradley was knocked down by a cruel line drive. In the second inning, Bradley faced off against Nolan Arenado, with Mark Wegner calling balls and strikes. What the numbers say is that Wegner messed up a 1-and-2 fastball. Let’s see what we have:
I went through the whole at-bat and, once more, PITCHf/x got confused. The pitch it thought was a 1-and-2 ball down the middle was a 1-and-1 strike a little above the middle, complete with a swing attempt. So for the fifth time, the pitch I was looking for didn’t exist. I mean, some of the pitches existed, but those that did didn’t exist as they were said to, in terms of the result. I investigated five called balls down the middle. I observed and confirmed zero called balls down the middle.
Which means, in the end, this is a post about nothing! That which it advertises, it doesn’t contain. I have found evidence of no called balls on pitches down the middle so far in the 2015 season. Now, because of this same kind of glitchiness, it’s possible there were such pitches, that just aren’t showing up. You can never rule that out, unless you happen to watch literally every pitch. But maybe look for the bright side here — everyone hates when umpires mess up. The worse the error, the worse the response. This is a post that searched for embarrassing mistakes, and it found that none of them were real. Way to go, umpires! There have been these ball calls in the past. It’s a good thing to not see them yet.
And it’s also a good reminder that PITCHf/x is far from infallible. Every so often, PITCHf/x goes and just makes something up. Now, it seems like human umpires do the same thing, and they do it way more often. That’s valid. Just know that the alternative to human umpiring isn’t perfection. The alternative to human umpiring is a system that, every now and again, would really freak people out about a potential future of automatically-driven cars.
Jeff made Lookout Landing a thing, but he does not still write there about the Mariners. He does write here, sometimes about the Mariners, but usually not.
This is hilarious. It’s basically a failed attempt at an article. But Jeff Sullivan, in his infinite glory, does not let that stop him from publishing it.
Precisely!
Negative results are still science! (And more seriously this is actually interesting data analysis even though it doesn’t say what he first expected it to say. People have a tendency to treat PitchFX as infallible because it _looks_ so precise and it’s good to be reminded just how much that isn’t the case.)
This probably being a large reason why MLB and the umpire’s union refuses to consider using computerized ball/strike calls or contesting the call on the field.
That’s exactly it, indeed!
Which is precisely why my fantasy for “robot umpires” leaves the umpire behind the plate doing the gestures and yelling the calls (and making safe/out calls on plays at the plate); it just adds a little cuneiform set of LEDs in his mask that tell him what PitchFX thought the pitch was (high/low, inside/outside, or in the zone). The umpire knows his performance is going to be judged against that data after the fact (it already is), so when he sees PitchFX wildly disagreeing with what he’s seeing then he can let the appropriate people know immediately that the system is borked.
But when he knows it’s working, he’s more likely to let it tell him what those borderline pitches actually were (and, perhaps more importantly, stop him from making the once-in-a-blue-moon hilariously bad calls that Mr. Sullivan was looking for when this article began).
You can’t stop Jeff Sullivan. You can only hope to contain him.
What I took from this is that Pitch F/X makes a ton of mistakes and I’m less sure of the quality of its data.
Over big samples, it’s solid. But whenever you see anything that seems weird, there’s no alternative to just pulling up the video to try to see it for yourself.
How do we know it’s solid? Has anyone looked at 1000 random videos of pitches and compared with the pitch fx recounting to see rates of error? I get that it is probably less than 10%, but it does matter whether it is 5%, 3%, 1% or .1%.
As I mentioned below, I worked for a company that produced pitching data for MLB games (and it’s a reputable company) using PITCHf/x data that was verified by watching video.
PITCHf/x is pretty solid for most data. It has trouble with outliers (like balls in the dirt, pitches significantly away from the strike zone, etc.).
With that said, PITCHf/x data is great for raw data (horizontal movement, vertical movement, location, velocity, etc.), but it is a model that will not quickly adjust to changes with the pitcher. So it’s classifications (i.e. “so what does the data mean?”) are prone to errors.
This is why Justin Verlander, according to PITCHf/x, in 2013 threw more change ups than fastballs.
He didn’t, but his velocity had dropped across the board, so PITCHf/x categorized it wrong.
In this case, we’re talking about what should be the very easiest calls: obvious strikes right down the middle. I’m pretty surprised by the number of screw ups.
We all assume pitch f/x is more accurate than humans. Do we really know if that’s true?
PITCHf/x doesn’t just measure x,y coordinates.
It also measures movement and velocities.
I think the later is far more significant–and PITCHf/x is pretty good with these measurements.
Humans probably can’t make these measurements.
But that’s not actually true, is it? These weren’t easy call and they weren’t strikes down the middle – they were BALLS down the middle, which are extreme outliers. Either the umpire screwed up or Pitch F/X screwed up – it’s not all that surprising that, given those two options, the Pitch F/X lost in all 5 instances of this small sample.
Good point, Neil.
PITCHf/x doesn’t just measure x,y coordinates.
It also measures movement and velocities.
This is absolutely wrong. All Pitch Fx does is measure ball positions. And it doesn’t measure ball positions anywhere close to the pitcher or home plate. It measures the position of the ball from about 45 feet from home plate until about 10 feet from home plate. It uses physics and these known positions to to predict the accelerations and velocities that describe an approximation of the balls entire trajectory path from the pitcher to home plate.
Although Pitch Fx data is not error free, most of what you are seeing in Jeff’s article art transcription errors that occur when Gameday attaches the Pitch Fx data to the game data recorded by its stringers. Any errors are usually corrected several days after the actual game.
Also, another note: sometimes, PITCHf/x corrects itself after the fact. If I recall, if you go back and watch Gameday, all the pitches above are as they should be. What that suggests is that the initial results were off, and then there’s some sort of second step, delayed, that edits the first.
This is definitely true. If you follow Gameday enough, you’ll notice plenty of errors that are correct later.
Sometimes Gameday will have the wrong batter up, and other issues (especially when something like Jean Segura stealing first base occurs), that are corrected later.
Given that five of five were wrong here you have the wrong IDs. If it were 2 or 3 even I might think that the pitch F/X data was bad, but 5 of 5 means that you are not doing a merge correctly or something like that.
I will also note that Wilson Ramos’ 5 ball walk caused gameday to pause for about 5 minutes and I’m not sure they ever got it in there correctly.
I used to work for an analytic company that produced pitch data on MLB games. We started with PITCHf/x data and then verified with footage of the game.
PITCHf/x has some serious location problems when pitches are in the dirt, or are far from the strike zone.
I found that the horizontal information was more likely to be correct in these instances than vertical information.
Also, velocities and pitch types were not as reliable on these type of pitches as with pitches that were closer to the strike zone (or in the strike zone).
Thank you. That doesn’t sound good at all.
Well, these aren’t frequent pitches. They’re outliers.
As Jeff mentioned above, the large data set is okay.
But sometimes flukes pop up like this.
Companies, like the one I worked for, attempt to reduce these errors by verifying like Jeff did.
If you’re concerned about PITCHf/x accuracy after reading this article, then you would be horrified to know how defensive metrics are accumulated.
Oh, these defensive metrics are something I would really like to hear about, especially if it involves John Dewan’s group, BIS. Please, THC, tell us more.
And, BTW, how many people here are actually old enough to remember catcher Harry Chiti. I am.
I could have sworn i saw one when watching the red sox game about a week ago. I think it might have been joe kelly on the mound, but i might be wrong. It would be way too much work to go through every pitch the red sox have thrown trying to find it, but i swear i saw it.
I currently work for a company that uses F/X data and sometimes it gets really off, and it’s not just pitch data. I did a Marlins game a few days ago where F/X didn’t notice that A.J. Ramos entered the game, instead skipping straight from David Phelps to Michael Dunn.
I did see what looked to be a down-the-middle pitch actually get called a ball in Tuesday’s Cubs-Pirates game, but that was because Wellington Castillo “framed” it 4 inches off the plate. That guy does pitch framing, like, backwards.
The pitch F/X mistakes aside. The biggest thing I took I article, is that umpires, are generally, very good.
There really isn’t data to support that in this article.
if you wanted to do automated balls and strikes you would put low energy IR lasers in the plate, pointing up, to get left/right and then do up/down with a camera designed for that purpose (not for tracking the ball’s entire flight and velocity).
Having been sitting with my eyes looking straight out at RHBs from the stands in the first few rows right behind the dugout, I can tell you that I’ve seen a pitch 4 or 5 inches below the knees called a strike. Umpires also penalize people who have good deception or really great sink b/c their pitches will look low to them when the catcher gets them out of the dirt. Basically, the zone is called not as if it was over the plate but where the catcher sits and that makes no sense.
You deduced that umpires are, generally very good, because they got correct 5 pitches that were right down the middle of the plate? OK.
I find that baseball data people (and I suppose data people in general) have a very hard time with the fact that their data is fallible. And while I agree with Jeff that it is reliable over large samples, it matters that it is fallible in specific cases (and it would matter if it was discovered that it contains biases in bigger cases, though that’s a separate issue). It puts into perspective not only automatic cars, as Jeff notes, but when technology is, and isn’t, the way to go.
Comparing F/X to self-driving cars is inappropriate. Putting buggy F/X software out in the world doesn’t endanger anyone’s life, so it’s fine to throw it against the wall and see if it sticks. Software to drive someone’s car will have vast amounts of testing, checks/balances, and regulations to guard against mistakes. There will likely be mistakes, but they’ll be fewer and farther between than F/X. “Technology” is not a monolithic entity. You can trust some and not others.
There’s a difference between the truth of technology and how people respond to technology. The average person is easily scared.
Which is why self-driving cars (on unrestricted streets, not on the carefully-vetted and minutely-mapped courses that have been demonstrated so far) are easily more than a decade away, technologically (from a regulations perspective, they may be even further). Despite what all the techno hype-miesters like to say.
Of course Yangervis Solarte is a Padre
I love the interest in pitch f/x, now can you please take a look at using it for a discussion of “batter framing”? All the love of catcher framing to get calls in their favor has totally glossed over the other side to it — hitters that seem to be able to work the umps to get more pitches in the zone called balls. I’m looking at you, Brian Dozier.
Here’s the baseball savant search of the players with most pitches in the zone that were called balls for 2014:
1. Dozier
2. Santana
Here’s the list so far for 2015:
1t. Dozier
1t. Santana
(suprise)
(Also relevant to this article Mr. Sullivan, check Dozier and you’ll see you missed the pitch on 4/26/15 down the middle called a ball!)
Do a search for any year and the same names pop up on top over and over. The same batters keep getting favorable calls from umps, it seems one of the most repeatable things going on in baseball right now. This isn’t just a case of “duh, obviously the best walk rate batters work the count”. This is a skill of getting strikes to be falsely called balls somehow. And Dozier was not a great eye before this — his walk rates were single-digits before he started getting “free balls”.
It’s such a massive advantage (Dozier’s OBA in ABs where he gets a free ball call is something like .500 OBA), I don’t know why it’s not getting any fangraphs attention.
Int-er-esting. Seems like something that would be right in Mr. Sullivan’s wheelhouse. Hope he looks at it.
Could be a function of the batter’s particular stance?
So maybe the Twins increase in BB% last year was from a few tips on how to stand in?
There was one in the dodger game the other night before mccarthy got injured. can’t remember what inning or what pitcher but I remember the announcers talking about it being a missed call and seeing the replay and it was right down the pipe.
Yordano Ventura has one last night that was almost dead center but was called a ball. The problem is that it was a 2-seam fastball that moved across the plate.
Jesse Hahn threw a curveball to Mike Trout last night (4/29) during his 1st AB that was down the middle and crossed the plate belt-high. The announcers were stunned that this strike was called a ball.. and blamed it on the Trout-effect.
Definitely over the plate, but at the belt isn’t quite down the middle, at least as far as middle-middle is concerned.
Mike Trout’s 2nd atbat today vs Jesse Chavez should qualify. took a 2-2 fastball that was middle-middle for a called ball, then went on to walk on the next pitch. I bet there are several examples from this game. This home plate ump is having a really rough day.
Looks like the pitch was up in the zone. In the zone! But up in the zone, so not quite “down the middle.”
There definitely was a down-the-middle pitch called a ball in that Oakland-Anaheim game. It was with a runner on, and the catcher stood up to throw either when the guy tried to steal second or a pickoff attempt – the pitch was right down the middle but called a ball because the catcher stood up before he caught the ball. Still, straight down the pipe, and the announcers (especially Ray Fosse, former catcher) lost it.
Wish I could help more than that, the ESPN pitch-by-pitch deal isn’t too helpful – probably in the bottom of the 6th? I seem to remember Alvarez was pitching.
Maybe someone with MLB.tv can find and gifify it – it definitely happened, just not on the pitch Pitch F/X says.
Here’s the inning I think it was from (Alvarez also pitched in the 7th) – http://espn.go.com/mlb/playbyplay?gameId=350421103&full=1&inning=6
I can’t find anything 🙁
Hmm. I know it was from that series but maybe I’m misremembering the pitcher. Second guess is from 4/20 – bottom of the 3rd as Trout was stealing second:
http://espn.go.com/mlb/playbyplay?gameId=350420103&full=1&inning=3
The A’s broadcast doesn’t do the strike zone overlay but they were super pissed and it definitely split the dish about thigh-high.
The wavefunction of a ball has tunneled into the strikezone, it appears.
Have there been any similar observations during games on ESPN? With the K-Zone on the screen live we should theoretically be able to get a photo of the ball crossing the plate in one location while the computer shows it in another spot.
To the article’s original question, check out the second pitch from Cole Hamels to JT Realmuto in the top of the 2nd on 4/22. Have never seen a catcher lunge at a ball down the middle like that, almost certainly costing him the call.