The Relative Importance of Fielding Metrics

Ultimate Zone Rating, created by Mitchel Lichtman several years ago and available free of charge on this very site, is a fantastic achievement in the world of baseball analytics, quantifying the logical intuition of what constitutes good or bad fielding by measuring the BIP-outs conversion rate in various different zones for each position. The data even breaks down into several different components, evaluating a player’s arm, ability to turn double plays, his ability to prevent runs by not making errors, as well as his range. One area of the data that gets overlooked far too frequently, though, that deserves to be mentioned more often, is that UZR calculates fielding abilities relative to the league average.

Sure, many of us know and understand that a +5 UZR means five runs better than the average player at that position, but grasping that the data is relative should force us to ask a secondary question upon glancing at a player’s results: did the league itself get better?

For instance, if a player turned 50% of the balls hit his way in a particular zone into outs in Year #1, when the league converted 40%, and then held stagnant at 50% in Year #2, when the league increased its conversion rate to 50%, the player didn’t change but his UZR would decrease. His actual overall ability to convert outs in that zone did not erode in any way, shape or form, per se, but his skills no longer looked as shiny because the talent level of the league at this specific position increased.

I like to refer to this as The Jimmy Rollins Conundrum, when his UZR marks hovered right around the league average in 2002 and 2005 despite playing a very, very solid shortstop. It could very well be that the eyes of myself and many other Phillies fans deceived us, with our scouting overrating Rollins’ fielding, but it seems that very few ever wonder if the fluctuations for a player in a given season are direct results of an improved league. Looking at Rollins in 2005, it is a bit tough to tell why the UZR fell to 0.8 from 4-5 runs above average the previous two years, before increasing to 6-7 runs over the next two seasons.

His double play runs were down as were his runs prevented by not making errors, but his overall number of errors in opportunities were consistent with the sandwiching seasons. Add in that his range was identical and that he would revert back to previously established norms in the double play and error runs departments and it stands to reason that perhaps one major reason for the lower UZR dealt with shortstops across the league improving in this area. I’m not suggesting this is the only reason, as we have seen fielders have down years before for one reason or another, but rather shedding light on a question we should be asking when looking at these numbers.

SO, moral of the story: don’t always assume that an increase or decrease in UZR is solely on the player, as the skill level of the league may have something to do with fluctuations as well.