Archive for uzr

Given the relative novelty of Statcast data, it remains unclear for the moment just how useful the information produced by it can and will be. As with any new metric or collection of metrics, it’s necessary to establish baselines for success. How good is an average exit velocity of 90 mph? What does a 10-degree launch angle mean for a hitter? How does sprint speed translate to stolen bases or defensive ability?

In an effort to begin answering such questions, the Statcast team has rolled out a few different metrics over the past few years that attempt to translate some of the raw material into more familiar terms. Hit probability uses launch angle and exit velocity to determine the likelihood that a batted ball will drop safely. Another metric, xwOBA, takes that idea a step further, using batted-ball data to estimate what a player should be hitting.

Another example is Outs Above Average. In the case of OAA, the Statcast team has accounted for all the balls that are hit into the outfield, determined how often catches are made based on a fielder’s distance from the ball, and then distilled those numbers down to find what an average outfielder would do. The final result: a single number above or below average.

At Reddit, mysterious user 903124 has published research showing that the year-to-year reliability for Outs Above Average has been considerably higher than the Range component for UZR. The user was kind enough (or foolish enough) to create a Twitter account reproduce some graphs of his results, which are shown below. There is a subsequent tweet in the thread that shows left field.

Recalculate correlation of RngR and OAA with per 1000 inning data and for each outfield position (min. 400 innings). OAA still clearly has a lower year to year variance. @mitchellichtman @tangotiger pic.twitter.com/MZ4xB6YICV

— 903124 (@903124S) January 3, 2018

 

 

For those who can’t see the charts and would prefer not to open up a new window, what you’d see here is that, for the group selected, the r-squared is much higher for Outs Above Average than for the Range component for UZR. If Statcast could produce something that is much more reliable and much more accurate than the Range component of UZR, that would be a pretty significant breakthrough and win for Statcast, potentially improving the way WAR is calculated and providing a better measure of a player’s talent and results on the field.

Read the rest of this entry »

This is now the third consecutive year in which I’ve written a post about the potential misuse of defensive metrics early in the season. We all want as large a sample size as possible to gather data and make sure what we are looking at is real. That is especially true with defensive statistics, which are reliable, but take longer than other stats to become so.

While the reminder is still a useful one, this year’s edition is a bit different. Past years have necessitated the publication of two posts on UZR outliers. This year, due to the lack of outliers at the moment, one post will be sufficient.

First, let’s begin with an excerpt from the UZR primer by Mitchel Lichtman:

Most of you are familiar with OPS, on base percentage plus slugging average. That is a very reliable metric even after one season of performance, or around 600 PA. In fact, the year-to-year correlation of OPS for full-time players, somewhat of a proxy for reliability, is almost .7. UZR, in contrast, depending on the position, has a year-to-year correlation of around .5. So a year of OPS data is roughly equivalent to a year and half to two years of UZR.

Last season, I identified 10 players whose defensive numbers one-third of the way into the season didn’t line up with their career numbers: six who were underperforming and four who were overperforming. The players in the table below were all at least six runs worse than their three-year averages from previous seasons. If they had kept that pace, they would have lost two WAR in one season just from defense alone. None of those six players kept that pace, and all improved their numbers over the course of the season.

The next table depicts the guys who appeared to be overperforming early on. If these players were to keep pace with their early-season exploits, the rest-of-season column would be double the one-third column. Brandon Crawford actually came fairly close to reaching that mark; nobody else did, however, as the other three put up worse numbers over the last two-thirds of the season than they had in its first third.

Just like with the underperfomers, all four of overperformers had recorded defensive marks six runs off their established levels. Replicating those figures over the rest of the season would have meant a two-win gain on defense alone. Again, no one accomplished that particular feat.

A funny thing happened when I ran the numbers for this season. There weren’t any outliers of a magnitude similar to last season or the season before. It’s possible you missed the announcement at the end of April, but there have been some changes made to UZR to help improve the metric.

Read the rest of this entry »

The Chicago Cubs are the unquestioned best team in baseball at the moment. There is no aspect of the game where the team struggles. They hit, hit for power, field and run the bases at a high level, pitch well as starters, and pitch well as relievers. When we ask questions and delve into the numbers, we do not ask if they are good. Instead, we ask how good are they, how this happened, and who is responsible. On the hitting side of things, numbers are easier to come by and believe in. On the run-prevention side, however, assigning value between pitching, defense, and luck can be difficult.

Back in June, August Fagerstrom noted that the Cubs’ opponent BABIP, then at .250, was basically the lowest of the past 55 years when adjusted for league average. Back in June, we had not yet completed half the season. Now in September, with the season nearly complete, the Cubs BABIP has risen… all the way to .251, increasing just one measly point. The Cubs are preventing balls in play at a record level.

On balls in play there are three principal groups of actors: pitchers, hitters, and defenders. While an individual hitter might have a decent amount of control over whether a batted ball becomes a hit or an out, pitchers face so many different hitters over the course of a season that, for any one pitcher and any one team, the control by the pitcher and defense on batted balls is likely very influential. So how do we break this down?

First, let’s back up a step, and note something else the Cubs have been doing at a historic level. Generally speaking, a team’s FIP is going to be fairly close to a team’s ERA. Since World War II, there have been 1,716 team seasons, and all but 108 (6.3%) have produced an ERA and FIP within a half-run of each other; two-thirds of teams, within a quarter-run. The Cubs are one of the biggest outliers we have ever seen.

So we see the Cubs up there, and wonder what could be causing this. Do the Cubs have a secret sauce? Is it the pitching? Is it the defense? Is this luck?

Read the rest of this entry »

Like any baseball stat, Wins Above Replacement provides the answer to a question. The question, in this case? Something like this: accounting for all the main ways (hitting, running, defense, etc.) in which a player can produce value for his team, how many wins has this particular player been worth?

There are, of course, criticisms of WAR. Some valid, others less so. One prominent criticism is how defensive value is handled in WAR. Some don’t understand how it’s calculated. Others understand but also question how well it represents a player’s defensive contributions. These criticisms shouldn’t be dismissed. As with all baseball statistics, though, it’s necessary to consider WAR in the context in which it’s presented — that is, to remember the question a metric is intended to answer and the method by which it attempts to answer that question.

On Monday, I completed one such reminder in a discussion of players whose WAR totals this year are probably low based on what we know about their defense. Today, I’ll make another attempt — this time, by examining players whose WAR totals are probably inflated by defensive numbers unlikely to be sustained over the course of a season.

In the comments of Monday’s post, one reader, Ernie Camacho, noted:

[T]here is a weird tension in this article between quantifying and estimating what has already happened, on the one hand, and evaluating player talent, on the other. I’m not sure we should be blending the two.

This is a good point. That tension most definitely exists, and it’s possible that some of that tension is what causes people to discount defensive metrics — and WAR as a whole. I agree that, in terms of calculating WAR, we should not be blending what has already happened with what we think will probably happen. Over time, in an ideal world, WAR captures both. In smaller samples, however, this is more difficult to do. In fact, there’s actually something that does capture the blending when we have smaller samples: projections. If we want to capture a player’s talent level at any given moment, projections do that very well.

Read the rest of this entry »

If you woke up this morning, looked at the WAR Leaderboards for position players and saw Mike Trout, Jose Altuve, and Manny Machado near the top, you might have had an inclination that all is right with the world. After all, those three players are some of the very best in major-league baseball, and we would expect to see them at the top of the list. Of course, when you look closely at the leaderboard, it’s important to note that there are 171 qualified players. To regard the WAR marks as some sort of de facto ranking for all players would be foolish. For some players, defensive value has a large impact on their WAR total, and it’s important, when considering WAR values one-third of the way into the season, to consider the context in which those figures.

“Small sample size” is a phrase that’s invoked a lot throughout the season. At FanGraphs, we try to determine what might be a small-sample aberration from what could be a new talent level. Generally speaking, the bigger the sample size, the better — and this is especially true for defensive statistics, where we want to have a very big sample to determine a player’s talent level. Last year, I attempted to provide a warning on the reliability of defensive statistics. Now that the season has reached its third month, it’s appropriate to revisit that work.

Read the rest of this entry »

Over the last calendar year, there are 139 qualified major-league hitters. Prorating their plate appearances to 600 per person, one finds that Mike Trout has the highest WAR at 7.2, followed by Russell Martin, Buster Posey, and Anthony Rizzo. None of that should come as much of a surprise, but the hitter right behind that group and just ahead of Josh Donaldson, Andrew McCutchen and Bryce Harper could provide a bit of shock. Over the last calendar year, Kevin Kiermaier has been worth six wins per 600 plate appearances.

Kiermaier, who has worked to improve his offense, is incredibly reliant on his fantastic defense for his great WAR numbers. While Kiermaier is a valuable player, it is possible that his WAR total is inflated by defensive numbers that are likely to come down over time. Kiermaier has logged roughly 1200 innings in the outfield and has a UZR/150 of 42.1, but only six active outfielders with at least 2,500 innings have a UZR/150 greater than 15, with Lorenzo Cain, Ben Zobrist, Peter Bourjos, Brett Gardner, Josh Reddick, and Jason Heyward falling between 16 and 22 — that is, roughly half Kiermaier’s current rate. Although he’s been good, Kiermaier is probably not the fifth-best player in baseball over the last year, and his defensive numbers should serve as a reminder that defensive statistics take some time before they become reliable.

Yesterday, I covered some players whose current WAR was potentially undervalued due to lower than normal defensive numbers in an article titled Heyward, Pedroia, and Your Annual Reminder About Defense. The present article renders yesterday’s title false as the articles together are now daily reminders, but this post should be the final one in this series with few, if any, more reminders coming in the near future. The caveat regarding small sample size from Mitchel Lichtman and our FanGraphs library is quoted more fully in yesterday’s piece, but to summarize: use three seasons of UZR when being conclusory about the defensive talent of any given player.

Read the rest of this entry »

We all know, entering the season, that the WAR leaderboards in the early part of the year reveal less about the players contained within them than those same WAR leaderboards at the end of the year. That knowledge doesn’t stop me, personally, from compulsively looking at the leaderboards just as soon as the season begins. Remember Freddy Galvis? He was tied for the National League lead among shortstops with 0.9 WAR — and “on pace” for a great season at the end of April. A month of replacement-level production has placed him considerably lower among major-league shortstops. What about Devon Travis? At the end of April, his 1.4 WAR was sixth in all of baseball. Unfortunately, an injury slowed him down and he has been unable to add to his impressive April totals.

Now that we have reached the second week of June, the leaderboards begin to look a little more familiar. Mike Trout, Josh Donaldson, and Paul Goldschmidt have continued great runs of production. Bryce Harper has emerged and Jason Kipnis has returned to form after a poor 2014 season. There are still surprises at this point, though. The production of Harper and Kipnis was not expected to reach these levels, Joc Pederson has been far more impressive than anyone could have expected, and Dee Gordon is still slapping and running his way into the top ten. We will see more changes as the season wears on, providing a more accurate depiction of player value as more games are played. However, since we are all looking at the leaderboards now, it might be worthwhile to point out a few anomalies in WAR totals due to the small sample sizes we have with defensive statistics.

Read the rest of this entry »

Like many wannabe saberdorks, I love Joe Posnanski’s work. It’s not just because he’s so much better than, say, [horrible-and-inexplicably-award-winning columnist for major newspaper] or [rumor-mongering baseball reporter prone to bouts of self-righteousness]. This isn’t a Posnanski tribute, but in short: Posnanski is great because he tells an engaging story and incorporates good baseball analysis without confusing one for the other.

This doesn’t mean that I always agree with Poz.* I disagree with many things written by sportswriters. In Posnanski’s case, I think highly enough of him that it’s worth quibbling over minor points, unlike, say, with [arrogant breaker of stories for your dad’s favorite sports magazine that we would have found out about anyway], who is only worth refuting because of his [alleged] influence. I hold Posnanski to a higher standard (not that he knows I exist).

*Or “JoPo”; has a sports journalist ever had so many different nicknames?

Which brings us to today’s Poz post on likely future Hall-of-Famers currently under 30. It’s an entertaining (if unsurprising) read. One claim in particular caught my eye. Posnanski writes that Ryan Zimmerman is “probably better defensively” than Evan Longoria. Now, Longoria didn’t qualify for the list (hasn’t played 500 major-league games), so while I do think he is the better player, that isn’t the point here. The issue is whether Zimmerman is “probably better defensively” than Longoria, as Posnanski claims.

Although he doesn’t cite specific defensive numbers in this piece, Posnanski has used Dewan’s plus/minus system in the past (although he has increasingly cited UZR). Here are the Dewan numbers for Zimmerman and Longoria in seasons in which they’ve both played (2008 and 2009):

Plus/Minus 2008:
Zimmerman: +10 plays (+11 runs) in 910.2 innings
Longoria: +11 plays (+9 runs) in 1045.2 innings

Plus/Minus 2009:
Zimmerman: +28 (+22 runs) in 1337.2 innings
Longoria: +21 (+17 runs) in 1302.2 innings

Over the last two seasons, Zimmerman has been 7 runs better in about 100 fewer innings according to plus/minus. Seven runs is seven runs, but given everything that is rightly said about the large error bars on defensive metrics, the gap isn’t as significant as it looks.

Given the various issues with defensive metrics, looking at other systems will give us a more perspicuous overview. Here at FanGraphs, UZR is used to measure fielding. I’m not qualified to argue which metric is the best; I’m simply using them as separate data points. UZR has a helpful “rate stat” version, UZR/150 (runs above/below average per 150 games). I’ve included the “non-rate” runs in parentheses.

UZR/150 2008:
Zimmerman: +3.4 (+2.1)
Longoria: +20.1 (18.5)

UZR/150 2009:
Zimmerman: +20.1 (+18.1)
Longoria: +19.2 (+14.9)

Suddenly things are less obvious. While 2009 was practically even, in 2008 UZR has Longoria almost two wins better. Their career UZR/150s: +12 for Zimmerman, +19.6 for Longoria. It’s a smaller sample for Longoria, but if you check Jeff Zimmerman’s regressed and age-adjusted 2010 UZR/150 projections, Zimmerman is at +10, and Longoria +12.

Defensive stats are obviously important, but when estimating fielding skill, in particular, we need to weight visual evidence — scouting — heavily. I’m not a professional scout, and unlike Posnanski, I don’t have access to them. Perhaps legendary scout Art Stewart, who told Poz “You will remember this day for the rest of your life” after Royals great Chris Lubanski’s first batting session at Kauffman Stadium, thinks Zimmerman is way better than Longoria. Jokes aside, scouting is essential for estimating defensive ability.

While most of us don’t have access to professional scouts, we do have access to the
Fans Scouting Report. In both 2008 and 2009 Longoria was rated as (slightly) better than Zimmerman.

Given that plus/minus seems to “prefer” Zimmerman — and UZR, Longoria — does this make the Fans Scouting Report a tiebreaker in Longoria’s favor? No. Given the relative closeness of the rating, neither the numbers nor the testimony of observers has the degree of reliability for us to make that kind of call. However, contra Posnanski, I do not think we can say that either player is “probably better defensively” than the other.

Molehill converted to mountain? Check. Happy American Thanksgiving, everyone!