Jackie Bradley Jr. and the Error Rule

In this week’s typically fantastic Sunday Notes column (if you don’t read them, you should), David Laurila passed along this tidbit which caught my eye:

Red Sox centerfielder Jackie Bradley Jr. was charged with an error on Friday night when he failed a handle a ball that, per StatCast, had a 6% catch probability. Bradley ran 60 feet in 4.2 seconds before having the ball carom off his glove.

That was backed up by David Adler.

Here’s the play in question:

Well, that doesn’t seem quite fair. But it’s becoming an increasing issue amongst official scorers and those who call for elimination of the error as a statistic altogether. Chris Hine, of the Minneapolis Star-Tribune wrote a great piece earlier this summer on different scorers’ approach to using Statcast: how much should Statcast data factor into the hit-vs.-error calculus? And do the Rules allow it?

Let’s start with the rule itself. Errors are governed by Rule 9.12 and its associated comments. The rule itself is almost five pages long in the official Rulebook and way too long to reproduce here. Further, the official definition of an error is no help at all: “An error is a statistic charged against a fielder whose action has assisted the team on offense, as set forth in this Rule 9.12.” Many have pointed to this language in the Comment to Rule 9.12(a)(1): “If a ground ball goes through a fielder’s legs or a fly ball falls untouched and, in the scorer’s judgment, the fielder could have handled the ball with ordinary effort, the official scorer shall charge such fielder with an error.” The Rulebook defines “Ordinary Effort” this way:

ORDINARY EFFORT is the effort that a fielder of average skill at a position in that league or classification of leagues should exhibit on a play, with due consideration given to the condition of the field and weather conditions.

The comments to this definition note that it’s an objective standard, not a subjective one.

[Ordinary Effort] is an objective standard in regard to any particular fielder. In other words, even if a fielder makes his best effort, if that effort falls short of what an average fielder at that position in that league would have made in a situation, the official scorer should charge that fielder with an error.

That means that effort should be measured relative to a reasonable player at that position, not to the player himself. Given this, it would seem that giving a fielder an error on a fly ball with a 6% catch probability is, in and of itself, erroneous. Statcast provides a reasonable, objective, fact-based criterion to apply to determine whether ordinary effort was supplied.

But part of the problem is that it’s not at all clear that an error is defined using the same standard as “ordinary effort.” Major League Baseball, on its own website, defines “error” this way:

A fielder is given an error if, in the judgment of the official scorer, he fails to convert an out on a play that an average fielder should have made.

This is the definition I’ve seen most cited on telecasts and broadcasts and at games. At first blush, that seems to be pretty close to the Rulebook definition. But there are some subtle differences. For example, it focuses on whether an out is made, rather than the effort exhibited; as a result, this definition is result-oriented rather than process-oriented. And something else from this definition merits some attention, as follows: “It simply penalizes a defender for not being able to make a routine play.” Well, that can’t be right. By definition, an average defender would make more than only routine plays. Statcast even has a metric, Outs Above Average, which uses catch probability to rank fielders. By that metric, Jake Cave has been an “average” fielder; his expected catch percentage (87%) and actual catch percentage (87%) are the same. Bradley, at nine outs above average, has converted more catch opportunities (90%) than expected (86%).

So let’s go back to the Rulebook definition. On the one hand, we have the objective standard created by “ordinary effort.” But in the comment, the language states that ordinary effort is in the judgment of the official scorer. And while perhaps those two standards could co-exist in the past, before advanced analytics and Statcast, that’s no longer the case. Why? Because Jackie Bradley Jr., by nearly converting an out on a fly ball with a 6% catch probability, delivered far more than “ordinary effort.” In other words, you can’t use the judgment of the official scorer to replace the fact-based metrics on which an objective standard relies.

The Comments to the Rule reinforce this disparity. Consider the comment on fly-ball errors:

The official scorer shall charge an outfielder with an error if such outfielder allows a fly ball to drop to the ground if, in the official scorer’s judgment, an outfielder at that position making ordinary effort would have caught such fly ball.

If the official scorer is supposed to be applying an objective standard, then it follows that they would benefit from referencing neutral, fact-based considerations (like Statcast) to determine what an average fielder is. But again, that’s not what the rule, which specifically says that it’s up to the official scorer, indicates. And, when viewed in that light, the meaning of the rule becomes clearer. “Ordinary effort” is theoretically an objective standard, yes. But at the same time, it seems as though MLB’s intent behind the rule was to make the official scorer the determinant of what “average fielding” is. It’s a standard which, again, might have made sense before the advent of Statcast. To describe a fly ball with a 6% catch probability as “ordinary,” however, is an exercise in absurdity.

Official scorers take a different view. Back in February, John Thorn, Official MLB Historian, passed along this explanation from official scorer Mark Jacobson:

Including these disparate plays distorts the scoring decision for the play at hand. Example (I’ve simplified Statcast for this, but just a bit): You see a fly ball. Statcast considers its opportunity time (available to a fielder running to make the catch) and the distance the fielder needs to cover to get to it…. If the Statcast database has 200 batted balls where the time and distance match yours — and 18 were caught — then the catch probability is 9% (18/200).

Now, you’re watching Billy Hamilton run down the fly ball. He reads it well, is amazingly fast, and gets there, well-positioned to make the catch — and clanks it. (When Statcast adds this to its database as #201, it’ll be a non-catch.) Now, you have to score it. You consider (at least)

• whether the fielder got to the ball (regardless of his speed, route, or judgment);
• whether he had a reasonable physical play on the ball when he got to it. (Was he well positioned? Was the ball too hot to handle? How about walls, sun, wind, etc.?)

While logical, the problem with the criteria being employed here is that it represents a subjective standard, not the objective standard required by the Rulebook. To show why, let’s consider the analogy that Jacobson does. If Billy Hamilton runs all-out and is well positioned to make a catch, the effort he took getting there will be a factor in his inability to catch the ball. A Billy Hamilton winded from running 30 yards at maximum speed will be less able to catch a fly ball as a result of the physical exertion involved in putting himself in that position — effort most center fielders couldn’t rival. And catch probability recognizes that. This isn’t to say that catch probability should be the arbiter of whether a fly ball is a hit or error. But it is to say that the fielder’s ability to place himself in a position to make a catch is relevant to making such a determination.

To be fair, defensive metrics have long been viewed — by authors at this site even — with some skepticism, much of it warranted. But it’s entirely unfair to say that defensive metrics have no use whatsoever. And using a person’s judgment to the exclusion of defensive metrics and Statcast data is more flawed than a scoring system based on data. But there’s an easy fix here — simply amend the comment to Rule 9.12 to say this:

ORDINARY EFFORT is the effort that a fielder of average skill at a position in that league or classification of leagues should exhibit on a play, with due consideration given to the condition of the field and weather conditions, as well as available data on comparable plays made under similar conditions.