What Mean Do You Regress Defensive Metrics To?

Jerry Crasnick has an excellent article on defensive metrics as they relate to valuing free agents, especially diving into how they affect Matt Holliday and Jason Bay. It’s no secret that, as the hosts of UZR, we’re big proponents of its usefulness. However, I still agree with essentially everything in Crasnick’s article.

There are aspects of defense that zone-based metrics won’t capture. There are results from UZR that make you scratch your head and say “really?” There is value in having the experienced eyes of a scout watch a player and offer an opinion on the abilities that he saw. We agree with all of that.

The cases where the value of metrics like UZR are the most contentious are when the results diverge significantly from what the perceived scouting wisdom says about a player. Often times, the reaction to counterintuitive data is to dismiss it entirely, offering up the example as evidence that the metric is flawed beyond use. Or, on the other side, to offer up the player’s numbers as proof that scouts just don’t get it, and that subjective opinions are worthless. Simply go back and re-read the threads about Mark Teixeira’s defense over the summer to see this effect in full force on both sides.

In reality though, both positions are wrong. Re-quoting the assistant GM from Crasnick’s piece:

“If there’s some kind of discrepancy, you need to use your best judgment,” the assistant says. “If a scout says, ‘This guy stinks,’ but the numbers say he’s excellent, the truth probably lies somewhere in between.”

This is essentially a paraphrase of the concept of regression to different means. If we have two players with identical UZRs, but scouts love one and abhor the other, our projection for their relative UZRs going forward should favor the one preferred by scouts. The fact that observational information is available gives us a useful data point to add to the calculation, pushing forward analysis that leads to “best judgment”.

I said last week that I think Teixeira is probably a bit better defensively than his recent UZR scores have indicated, and the foundation of that belief lies in the value of scouting information. Teixeira is revered by almost every scout in the game as an exceptional defensive first baseman. That matters when we’re projecting future defensive performance. There is no reason to simply ignore those opinions simply because they don’t line up with what UZR has measured. We account for those opinions by regressing Teixeira’s UZR projections to a different mean than a player that scouts are less enamored of.

UZR is a tool. Scouts are a tool. They can be used together to produce better information than either can on their own. It is not an either/or proposition. Use both.