Some Thoughts on Batted Ball Data

Colin Wyers wrote a post today about potential bias in batted ball data. While I don’t have anything in particular to say about the results of his bias study, I have to disagree with his conclusion and debunk some of the information provided about the differences between Stat Corner tRA and FanGraphs tRA, which he uses to illustrate his point:

For starters, the difference in tRA between FanGraphs and Stat Corner is a poor stat to illustrate GB/FB/LD bias because there are other differences in the way both sites calculate the stat. Let’s take Felix Hernandez this year, for whom BIS and Gameday have very, very similar batted ball profiles for 2010.

         GB      LD     FB 
BIS     67.6    13.5   16.6 
GD      65.8    13.2   15.8

Now, here’s the difference in FanGraphs tRA vs StatCorner tRA

FG – 4.62
SC – 5.05

Almost a half a run difference. Why are they so different? It’s probably the component park factors, mainly on LD% and HR%, I would imagine.

Actually, I’ll plug both of those stat lines into the FanGraphs tRA calculator and see what I get: 4.62 and 4.70. So, about .08 of the differences is because of GB/FB/LD differences and the other .35 is park factors (or potentially slightly different weights).

Furthermore, if you look at individual player GB% correlation from 2003 to 2008 between BIS and Retrosheet data, you get .94. That’s among all players, whether they pitched 1 inning or 200 innings. Here’s the others:

GB% – .94
FB% – .85
LD% – .72

It’s not like the two data sources are telling you completely different things. For the most part, they agree, especially on GB%.

Baseball Info Solutions also rotates their scorers, to try and avoid any scorer bias as Ben Jedlovec stated here:

BIS Scorers are assigned “randomly”. We’re not using a random number generator, but it’s almost as effective. Scorers have a designated number (Ex. Scorer #11) which are then rotated through different slots in the schedule. If scorers 7 and 8 are scoring the late (west coast) games one day, they’ll be rotated to early games the next time around. There’s some miscellaneous switching to accommodate vacation, etc. too. In the end, everyone’s getting a good mix of every team in every park.

We also have several different quality control methods in place to make sure that scorers are consistent with their hit locations and types. We added some new tests this season using the hit timer to flag the batted ball data, so the 2009 data is better than ever.

Ben continues with:

BIS gets an almost entirely new set of video scouts each season. If you’re seeing the same “bias” in the same parks year after year, I can’t see how it would be related to the individual scorer.

It’s also important to note that BIS has an additional classification of batted ball data, Fliners, which is not displayed on FanGraphs and lumped in with Line Drives and Fly Balls. Fliners come in two varieties, Fliner-Line Drives and Fliner-Fly Balls.

Colin tackled the line drive issue before on the Hardball Times, in which Cory Schwartz of MLBAM responded:

our trajectory data is indeed validated as thoroughly as all of our other data: not just once, but three times: first, by a game-night manager who monitors the data entered by the stringer, second by a next-day editor who reviews trajectories against video, and third by Elias Sports Bureau. We take great care in the accuracy of all our data, including trajectories.

None of this is to say that your original premise is not true: line drive vs. fly ball is indeed a somewhat subjective distinction that may be influenced by a number of factors, not just press box height. But I disagree with your assertion that the accuracy of our quality is inferior in this (or any other) regard.

Now we know that there is subjectivity in batted ball stats, but in Colin’s conclusion he writes:

In the meantime, consider this my sabermetric crisis of faith. It’s not that I don’t believe in the objective study of baseball. I’m just not convinced at this point that something dealing with batted-ball data is, at least wholly, an objective study. And where does this leave us with existing metrics that utilize batted-ball data? Again, I’m not sure.

For me, this is a bit of an extreme conclusion to make. For stats like GB% I think there is little to be concerned about, but once you get to LD%, I think you should realize there is some subjectivity involved. Is it worth disregarding entirely or having a “sabermetric crisis of faith” over? In my opinion, probably not.

We all want best data possible and there are some exciting projects underway to collect more granular and precise data, but in the meantime, I don’t see any reason to dismiss the data that is currently available. Better batted ball data will certainly lead to more accurate results; I don’t think it will show completely different results.

Authors Note: This was an expansion on my thoughts from a comment I posted on insidethebook.com