Heyward, Pedroia, and Your Annual Warning About Defense

We all know, entering the season, that the WAR leaderboards in the early part of the year reveal less about the players contained within them than those same WAR leaderboards at the end of the year. That knowledge doesn’t stop me, personally, from compulsively looking at the leaderboards just as soon as the season begins. Remember Freddy Galvis? He was tied for the National League lead among shortstops with 0.9 WAR — and “on pace” for a great season at the end of April. A month of replacement-level production has placed him considerably lower among major-league shortstops. What about Devon Travis? At the end of April, his 1.4 WAR was sixth in all of baseball. Unfortunately, an injury slowed him down and he has been unable to add to his impressive April totals.

Now that we have reached the second week of June, the leaderboards begin to look a little more familiar. Mike Trout, Josh Donaldson, and Paul Goldschmidt have continued great runs of production. Bryce Harper has emerged and Jason Kipnis has returned to form after a poor 2014 season. There are still surprises at this point, though. The production of Harper and Kipnis was not expected to reach these levels, Joc Pederson has been far more impressive than anyone could have expected, and Dee Gordon is still slapping and running his way into the top ten. We will see more changes as the season wears on, providing a more accurate depiction of player value as more games are played. However, since we are all looking at the leaderboards now, it might be worthwhile to point out a few anomalies in WAR totals due to the small sample sizes we have with defensive statistics.

This might be old hat for some, but it’s important to keep in mind that defensive statistics are not tremendously reliable after just a few months when trying to determine the talent level of the fielder. In his UZR Primer, Mitchel Lichtman compares the reliability of UZR to OPS.

Most of you are familiar with OPS, on base percentage plus slugging average. That is a very reliable metric even after one season of performance, or around 600 PA. In fact, the year-to-year correlation of OPS for full-time players, somewhat of a proxy for reliability, is almost .7. UZR, in contrast, depending on the position, has a year-to-year correlation of around .5. So a year of OPS data is roughly equivalent to a year and half to two years of UZR.

Our FanGraphs library provides the following caveat:

Beware of sample sizes! If a player only spent 50 innings at a position last season, it’d be a good idea not to draw too many conclusions from their UZR score over that time. Like with any defensive statistic, you should always use three years of UZR data before trying to draw any conclusions on the true talent level of a fielder.

With that in mind, I attempted to find players whose current WAR totals might be weighed down by unusual defensive numbers early in the season compared to what would generally be expected given what we know about a player’s talent level. To find appropriate players to study, I looked at only those players who were qualified hitters from 2012 to 2014, who did not change positions, and who are also qualified this season. These 70 players all accumulated at least 2,500 innings on defense in the previous three years, with some players having significantly more time in the field. While young players and players who have changed positions could have some unusually high or low UZR numbers affecting their WAR, those players do not have the track record to appropriately compare prior numbers.

To get a general determination of how far the 2015 numbers were diverting from the previous three seasons, I took the three-year defensive runs, divided by three to get to one year, and then divided by three again (also called dividing by nine), as we are one-third of the way through this season. I compared that number to the current season’s defensive-runs number. Most players were relatively close to previous years, with 60% of the 70 players within three runs of prior years and 80% of players within five runs. Below are some of the players who appear to have been hurt the most in WAR by lower than typical defensive numbers. For each player, the seasonal average is used for 2012-2014, and Def includes the positional adjustment.

Dustin Pedroia

Dustin Pedroia DRS UZR Def 1/3 Season Def
2012-2014 AVG 14.3 13.0 15.2 5.1
2015 0.0 -1.4 -0.6 -0.6
DIFFERENCE -5.6

At 31 years old, Pedroia is likely in the decline phase of his career but the odds of him decling from one of the best second baseman in the game to slightly below average seems unlikely. Pedroia won the last two Fielding Bible Awards and three of the last four. He has hit very well this season, with a 133 wRC+ placing his WAR at 1.8 for the season, but it looks like he might be half a win better than his current WAR represents.

Ian Desmond

Ian Desmond DRS UZR Def 1/3 Season Def
2012-2014 AVG -2.3 3.0 9.7 3.2
2015 -4.0 -5.2 -2.6 -2.6
DIFFERENCE -5.8

Desmond will never be confused for a defensive star like Pedroia, but his numbers this season are worse than normal for Desmond. DRS and UZR disagree by about five runs per season, and averaging the two would render Desmond a roughly average shortstop, but the numbers this season are much worse. Desmond is having a disappointing season at the plate as well, giving him replacement-level numbers after three straight four-win seasons. Desmond is not playing well this season, but his replacement-level WAR is likely not as bad as it should be.

Jason Heyward

Jason Heyward DRS UZR Def 1/3 Season Def
2012-2014 AVG 22.3 20.0 14.2 4.7
2015 4.0 0.9 -1.4 -1.4
DIFFERENCE -6.1

Like Pedroia, Heyward has an excellent defensive reputation, winning the Fielding Bible Award for right field in 2012 and again last season. Unlike Pedroia, Heyward has not gotten off to as good a start with the bat, hitting .217/.261/.349 in April. He got better results in May with a wRC+ of 115, but his WAR still sits at 0.3 due to his poor April and defensive numbers that might not represent his talent.

Talent that allows him to do this, that is:

Denard Span

Denard Span DRS UZR Def 1/3 Season Def
2012-2014 AVG 6.7 4.4 6.5 2.2
2015 -4.0 -5.1 -4.5 -4.5
DIFFERENCE -6.7

Denard Span has generally been an above-average center fielder during his career, although there is a little less confidence regarding Span’s talent level on defense compared to the players above him on this list. Span’s negative numbers last season by themselves would not really mean a whole lot, but taken with this year’s numbers, his age (31), and the fact he had knee surgery in March might mean that his 2015 defensive numbers are closer to reality than we might think. Even with those concerns, it’s unlikely that Span is actually close to 20 runs below average over the course of a full season as his current UZR might make him seem.

Chase Headley

Chase Headley DRS UZR Def 1/3 Season Def
2012-2014 AVG 5.0 11.7 13.9 4.6
2015 -8.0 -6.4 -5.5 -5.5
DIFFERENCE -10.1

Headley is another player who’s a little bit tough to figure out right now. He has an excellent defensive reputation, but he has also made seven throwing errors on the season, a problem he is working on. Headley is likely to be good going forward, but given his throwing problems, which did not seem to be present previously, it is likely that he has not been as good defensively as in prior years.

Looking at leaderboards can be an enjoyable experience, but, even at this stage of the season, there is a lot of movement remaining before the end of the year. It’s easy to look at a player’s WAR right now, multiply by three, and say they are on pace for an x-win season, but given defensive sample sizes, doing so can misrepresent the likely true-talent levels for players like Pedroia and Heyward.





Craig Edwards can be found on twitter @craigjedwards.

37 Comments
Oldest
Newest Most Voted
Alex
11 years ago

Not saying I don’t believe in the math, but I’m always shocked to see Denard Span labeled as anything other than “stellar” on defense. I’ve loved having him in center field and he always seems to run down everything with a great route. Definitely a case of the “eye test” disagreeing with the sabermetric data.

Greg
11 years ago
Reply to  Alex

Hey Alex, one of his best skills has always been his range, but he is notorious for having one of the weakest arms in the MLB. Now that he’s posted 1.3 years of poor range, it’s not all that surprising. 30-40% of defense is holding runners and throwing people out, and if you’re below average there, you need to be elite on the other 60-70 % to make up the difference.

Pig.Pen
11 years ago
Reply to  Alex

I wouldn’t say the math doesn’t back it up, I’d say that UZR doesn’t back that up. DRS seems to agree with the eye test and while UZR doesn’t say he’s bad, it doesn’t say he’s elite either. One of the many reasons that more and more I believe UZR is a junk stat that does more harm than good. [see also: Harper, Bryce]

Candy LaChance
11 years ago

This is sort of a weird piece. Yes, defensive metrics through 1/3 of the season are not very predictive, so are particularly likely to regress towards career norms. But offensive stats this early aren’t necessarily predictive either, even if better than defesive.

If the goal is estimate true talent, isn’t the best answer to look at the projections?

tehzachatak
11 years ago
Reply to  Candy LaChance

as i understand it (caveated: my understanding may be fully wrong) UZR isn’t really a good measure of “what happened,” which is what i think Craig’s alluding to. it’s not a perfect (or probably even that good) of a measure of what actual defensive value has been in a smaller sample.

therefore, when we look at Pedroia or Heyward’s fWAR to date, it’s not necessarily particularly representative of what their value has actually been to date, especially if a large positive or negative component is attributed to defense. a more representative estimate of what their value actually has been might take their offensive runs, positional adjustment, replacement value, and add in a prorated version of their career average defensive runs, or average defensive runs over the past ~3500 innings.

Cool Lester Smooth
11 years ago
Reply to  Candy LaChance

The difference is that BIS data is far less reliable than box score data, and that UZR/DRS/PM are far less accurate models than wRAA.

It’s a double whammy of “don’t trust this in small samples.”

Stephen BrownMember since 2020
11 years ago

Craig, great article! I came up with a method for doing this a while back that you may be interested in. Read about it here: http://bravesgeneralstore.com/regressed-war-2015/

Here’s the latest leaderboard for it:
https://docs.google.com/spreadsheets/d/1DSp2oiLaay-EcOcZKh1VtdNx2Hq3ucgjQNaLe3MPg2k/edit?usp=sharing

Stephen BrownMember since 2020
11 years ago
Reply to  Stephen Brown

At some point I will start using my new Expected Slash work for 2015, rather than the current BABIP and HR/FB adjustments: http://www.fangraphs.com/community/using-batted-ball-data-to-measure-hitter-performance/

Lucas
11 years ago

Brett Gardner is also notable.

DNA+
11 years ago

To me, this highlights a lot of the silliness of the defensive numbers. Chase Utley is hitting well below what anybody has expected him to hit. But no one would argue that his real offensive value has been better than what we have measured because we all know that the bad outcomes have actually happened. Here, you are arguing that Pedroia has actually been worth more defensively than what you have measured because you just know he is a good defender so he must have played like a good defender. The only reason you can make this argument with a straight face is because no one actually knows what has happened. But if we know what the answer should be before we start, what is the point?

redsoxu571
11 years ago
Reply to  DNA+

What you are ignoring is the relative lack of precision in defense metrics over a small sample size.

Remember when you first learned in school the difference between accuracy and precision? Precision is how tight the results will cluster, and accuracy is how on-point that clustering (tight or otherwise) will be to the “truth”.

Defensive metrics seem to be very accurate (hence their value), but they also seem to be rather imprecise. Three years is a long time to reach statistical stability in the baseball world. That means that any one or two month result has two much of a potential spread.

Anyone who has watched Pedroia on a daily basis would agree that his defense hasn’t quite been up to snuff this year, so there likely has been some drop. But it also doesn’t look like as big a drop as the metrics show, suggesting that the accurate point around which the results are trying to cluster is better than the numbers produced right now.

DNA+
11 years ago
Reply to  redsoxu571

Why the needless condescension?

I am not ignoring the lack of precision at all. I am highlighting it.

BenRevereDoesSteroids
11 years ago
Reply to  redsoxu571

How do we know that UZR/DRS are accurate? We can tell that with something like wRC because there is a near 1 to 1 correlation between wRC and runs scored. But where is the evidence for UZR?

evo34Member since 2023
11 years ago

Exactly. The fact that some unproven variable correlates to itself fairly well y2y (0.50) is meaningless when it doesn’t correlate fully with run prevention. OPS, on the other hand, correlates almost 100% to runs scored, so it’s very important to measure.

The article should be shortened to say: don’t ever even glance at in-season UZR.

Shao
11 years ago

I kind doubt UZR and DRS have some issues dealing with shift. According what they claim, no shift play will be taken in to account when calculating. However, if you check the OOZ vs BIZ year over year, you will see OOZ per BIP trends way above BIZ per BIP. Or put it in other words, in 2010, you will see an OOZ every 30 inning ,and now in 2015, it only takes you 24.

I doubt there are any reasons the overall defense improve that much in just 5 years. I think it has something to do with shift. Maybe DRS/UZR doesn’t identify not-so-obvious shifts right. Like Jung-ho Kang, I’ve watched every single game he played this year, and it’s very difficult for me to believe he has 10 OOZ at third in just 140 IP. Considering the fact the Pirates uses lots of shifts, I suspect BIS mistakenly count shifted play as OOZ.

Maybe you can run a simple test to see if a team’s OOZ has positive relation with total number of shift that team uses. Or, even simpler, ask BIS how they define shift.

Moonbruin
11 years ago

Hi Guys,
Would you start Desmond over Correa this week??

DNA+
11 years ago
Reply to  Moonbruin

They play on different teams so it isn’t really an option for either manager.

mosc
11 years ago
Reply to  Moonbruin

yes

Sandy Kazmir
11 years ago

if you watched baseball you would see that Pedroia has looked lime complete ass in the field. He has lost a step laterally and maybe more than one.

Beard!
11 years ago
Reply to  Sandy Kazmir

Why don’t you keep your drivel to your Rays blog.

Mike
11 years ago
Reply to  Beard!

I usually hate comments like this, but yeah. The Pedroia statement has so much more to do with its author than anything that actually occurred on any baseball diamond. Alas for not being able to pimp Ben Zobrist any more, I guess?

Beard!
11 years ago
Reply to  Mike

Exactly, if it wasn’t Hanselman, and if he had commented about any of the other players mentioned along with Pedroia, then maybe it wouldn’t be a worthless post. However, when the guy who makes a comment is a guy who almost solely posts Red Sox hate on every article they’re mentioned, well that’s when the post becomes worthless.

Sandy Kazmir
11 years ago
Reply to  Tim

HE’S FIXED! If you aren’t concerned about his lack of range this year then you’re not watching him play or you’re a fan of the opposition and smiling more than the cat batting a mouse.

Jim Lahey
11 years ago

I think Desmond has actually earned his sucky defensive ratings this year. Same for Headley.

Don’t really understand the other 3 though.

DNA+
11 years ago
Reply to  Jim Lahey

Headley certainly has. He has been fantastic as always at getting to balls, but he has just thrown too many outs away. I hope it doesn’t develop into a real case of the yips for him.

Chris
11 years ago
Reply to  DNA+

Agreed. He is still making the tough/low percentage plays average third basemen cannot, but he routine plays (especially throws) have been a big issue for him this year. I think Teixeira has saved a few more errors for him on what should have been routine plays/throws as well.

Cool Lester Smooth
11 years ago
Reply to  DNA+

Yeah, hopefully he hasn’t gone full Zimmerman.

Gai1997
11 years ago
Reply to  Jim Lahey

Headley continues to botch throws and boot routine ones, and then proceed to make an incredible diving play to rob a hit. Clearly he hasn’t fallen off a cliff, if anything it’s mental.

redsoxu571
11 years ago

I can’t cite exact numbers and dates, but I remember investigating defensive numbers fairly early last season (maybe 6 to 8 weeks in?), and it stood out to me that Dustin Pedroia’s numbers were pretty average (and therefore low for him) then too. So if anyone wants to call BS on the argument of this article, I would like to note that one of these same players seemed to be experiencing similar results last year (they were better last year, but then again Pedroia seems to not quite be playing at the same defensive level this year anyways). Chalk it up as part of the nature of the beast, and we’ll check in on all these guys when the season hits the stretch run.

vivalajeter
11 years ago

Juan Lagares is another example. He doesn’t make the cut-off because he was in the majors in 2012, but his defensive numbers have plummeted. He went from being the best CF in baseball, to being right around average.

Part of it might be a true lack in defense, because he’s been dealing with some injuries. But part of it is opportunities. He’s played about 1/2 the innings this year as he played last year. But if you look at Inside Edge data, he has 280 ‘routine’ chances last year, and only 110 this year. Maybe the Mets’ staff isn’t giving up as many fly balls to center – and with Harvey back, it makes sense that there would be fewer balls in play due to the uptick in strikeouts – but the Inside Edge percentages are still solid for him.

senor_mike
11 years ago

Juan Lagares is suffering from the same problem, thought he probably isn’t as noticeable because his defensive numbers are slightly positive and/or neutral.

He most definitely continues to pass the eye-test for me, and FWIW Inside Edge believes he is still an animal this year as well.

Other than that, I agree with Jim Lahey that Desmond’s defensive value/performance/etc. up to now IS being reflected accurately in his rate numbers. He was pretty terrible in the field in month one, we should not just ignore that because he’s been better historically than what April tells us.

phoenix2042
11 years ago

Headley, for whatever reason, has already matched his single season career high in errors and we’re only a third of the way through the season. Don’t know if his true talent has changed (I mean I understand losing a step or arm strength, but does aging cause lack of accuracy?), but he has certainly played worse this season and deserves his negative rating.

mike
11 years ago

Doesn’t Headley have something like 12 errors already this season (way more than he normally would)? Maybe Desmond and Pedroia are having some similar experiences and that’s part of why their UZR is weighed down?

So maybe their current WAR is somewhat valid but we should expect that they perform better moving forward by limiting errors?

Just spitballing here.

Gai1997
11 years ago
Reply to  mike

Don’t know about Desmond, but for Headley 100% yes. He still looks like the same defender last year, he’s making the same plays to rob hits, but he’s failing when it comes to routine ones and throws.

brendan
11 years ago

Hi Craig, when you say this:

” …but it looks like he [pedroia] might be half a win better than his current WAR represents.”

I’m not sure this is what MGL means. As I understand it, 1/3 season on UZR is not predictive, but it’s not necessarily _wrong_. That is, I think that pedroia in fact did _not_ make some plays and hurt his team. He may very well play better defense the rest of the year, but his WAR should be considered to have captured his defensive contribution so far.