It’s that time of year again! Tangotiger is running his annual Fans Scouting Report. Please take a moment and fill out scouting reports for the team of your choice!

Occasionally (okay, rather frequently), I’ll see people debate the accuracies between the WAR displayed on FanGraphs and Rally’s WAR on Baseball-Reference.

Joe Posnanski speculated on the differences in a recent article about Josh Hamilton’s MVP chances:

*I could be reading this wrong, but Fangraphs seems to put more emphasis on defense. For instance, Carl Crawford’s WAR at Baseball Reference is 3.7 — his defense is worth eight runs above average. But Fangraphs credits him for 22 runs above average, which thrusts his WAR up to 5.6 and into the No. 4 spot in baseball.

I’ve seen similar sentiments echoed throughout the blogosphere and on Twitter.

In reality, on a per-player basis in 2009, UZR distributed 441 fewer runs than TZ did, excluding pitchers and catchers. And there is not a year that UZR is available where its absolute value has been higher than TZ.

In 2009, the maximum spread of UZR was +31 to -37 and TZ showed a similar spread of +31 to -34. Here’s a graph of the full spread. The blue overlap shows the points at which TZ starts showing a greater spread.

This might not be a perfect comparison in how much defense actually contributes to WAR and a better one might be how much as a whole does fielding contribute to total runs. In 2009, fielding made up about 14.2% of all positive and negative runs according to FanGraphs WAR, while Rally’s WAR made up about 15.5% of all positive and negative runs.

All in all, they are similar in how fielding is weighted as a whole. The biggest difference between the two is how each individual player’s fielding is evaluated.

Update: Tangotiger pointed out to me that fans scouting report numbers for Holliday and Wilson looked off. Turns out I was calculating players who switched teams for the Fans Scouting Report incorrectly and as a result Matt Holliday and Jack Wilson are now adjusted accordingly and the tables below have been updated.

Earlier this week I ran the the Aggregate Defensive Evaluations (ADE) on 2009 shortstops. For those who missed it, this is an attempt to take 5 different fielding metrics (UZR, Fans Scouting Report, John Dewan’s DRS, Total Zone, and Total Zone with Location), put them on the same scale and then see which player’s defensive abilities we are fairly certain about and those which we are not.

In response to some comments, I’ve added a weighted average and standard deviation. This excludes standard Total Zone in favor of TZL. It also weights UZR, DRS, and TZL 3 times each and then the Fans Scouting Report only 1 time. (These are the last two columns)

It’s probably not much of a surprise that Carl Crawford sits atop the list. UZR has actually rated him as 56 runs above average the past three years, more than double the next closest player. Same goes for pretty much all the defensive metrics.

Matt Holliday I would say has the highest level of disagreement of any player. The Fans Scouting Report hates him, DRS loves him, UZR thinks he’s above average, and Total Zone thinks he’s just average. I’d consider the inconsistency with Holliday different than Juan Rivera’s situation where there’s also a high level of disagreement. In Rivera’s case at least all metrics agree he’s average or better.

Ryan Braun is also pretty interesting in that the Fans rated him as +15, while all the other defensive models thought he was well below average. This season Ryan Braun continues to be rated poorly by the defensive models. It will be interesting to see what the fans think of him next season.

I’d still consider these reports a bit of a work in progress, but for those interested, here’s the shortstops again, but this time with the weighted averages column:

There’s no denying defensive metrics are controversial. Whether they clash with what you’ve seen with your own eyes, or you just don’t believe them, it seems like everyone has some sort of opinion to offer on their validity.

On FanGraphs, we carry no less than four different defensive metrics:

UZR – Mitchel Lichtman’s Ultimate Zone Rating
DRS – John Dewan’s Defensive Runs Saved
TZL – Rally’s Total Zone (location based version)
TZ – Rally’s Total Zone (standard version)

There’s no denying that we use some more frequently than others (cough, UZR), but the reason we have all four is because it’s great to see what different data sets and different models spit out. And In addition to the four, there’s also a fifth completely unrelated metric in the Fans Scouts Report that is run each and every year on insidethebook.com by Tangotiger.

It’s important to note that all these defensive metrics are not on the same scale, so it’s difficult to glance at all four (five if you use the Fans Scouting Report) and get a good sense if they’re in agreement or not. Which brings me to the preliminary look at the Aggregate Defensive Evaluations, where each metric is put on the same scale for each position, averaged, and then a standard deviation is computed for each player. Here are the 2009 Shortstops (min 82 games played):

As you can see, Paul Janish and Brendan Ryan are the clear leaders atop the list and even all the metrics are for the most part in agreement. +/- 5 runs in either direction is still going to make them elite defenders.

And there are players like Yunel Escobar who is considered by Total Zone and DRS to be very good, but by UZR and the Fans to be more or less average. On an aggregate level he still ends up as very good, though there is a good amount of disagreement as to just how good he is, even if no system thinks he’s below average.

All in all, it should be easy to go up and down the list and see which players there’s a high level of confidence about defensively, and which there is not.

From a mere computational standpoint, is this the best way to go about combining defensive metrics? I’m really not sure and it’s certainly worth looking into further. There are a lot of options in weighting the metrics differently and how to scale them, but overall I feel this is at least a decent start and something I hope to delve into a bit more.

The point here is that there’s a lot of information in these metrics with so many models out there it’s becoming increasingly important to try and identify what we’re fairly confident about and what we’re not so confident about instead of making the mistake of throwing them all away.

Good news everyone!

Minor league stats are updated for 2010 and will continue to be updated throughout whatever’s left of the season. We also have added the 2010 Mexican League, Venezuelan Summer League, and Dominican Summer League and plan to have the the winter leagues as well.

There’s still a little bit of cleanup that needs to be done in terms of player matching (Strasburg for instance has 2 pages still), but that should be completed this weekend.

Just 1 day until the event! Tickets will be available online up until 8am tomorrow morning as long as they’re not sold out.

Location: Florence Gould Hall (55 East 59th Street)

Tickets: $15 online (+$1.36 surcharge). $20 cash only at the door.

NY Baseball (9:00am – 9:40am)

Joe Pawlikowski, Mike Axisa, Benjamin Kabak (All RiverAveBlues.com), Matthew Cerrone (MetsBlog.com), and Mark Simon (ESPN) will be discussing all things baseball in NY. Moderated by Carson Cistulli.

Baseball Media (9:45am – 10:30am)

Jonah Keri (Bloomberg Sports) will host a panel comprised of Will Leitch (Deadspin, New York Magazine), Michael Silverman (Boston Herald), Matthew Cerrone (MetsBlog.com), Alex Speier (WEEI.com), and David Biderman (WSJ) to discuss how baseball media coverage has changed in recent years and will continue to evolve.

Baseball Stats (10:40am – 11:15am)

Jon Sciambi (ESPN), Mitchel Lichtman, Sky Kalkman (Beyond the Boxscore), Dave Cameron, and David Appelman will discuss where advanced baseball stats are right now and where they’ll be headed. Moderated by Carson Cistulli.

Bloomberg Sports Presentation (11:20 – 11:35)

Bloomberg Sports will make a presentation of a brand new product.

FanGraphs Q&A (11:40 – End)

Dave Cameron, Carson Cistulli, Bryan Smith, Joe Pawlikowski, Mike Axisa, and David Appelman will take questions until we’re officially kicked out (a little after 12:00).

Afterparty (3:30pm – Game Over)

Additionally, we’re going to host a game-watching party for attendees to gather at a local watering hole and view that afternoon’s Boston-New York match-up together. Those who make it to the event will be invited to join us for several more hours of fun later in the afternoon. Details and directions will be given at the event.

For those of you who are still making up your minds about whether or not to come to the very first FanGraphs live event in NYC this Saturday, August 7th at 9am, here are the full details:

Location: Florence Gould Hall (55 East 59th Street)

Tickets: $15 online (+$1.36 surcharge). $20 cash only at the door.

NY Baseball (9:00am – 9:40am)

Joe Pawlikowski, Mike Axisa, Benjamin Kabak (All RiverAveBlues.com), Matthew Cerrone (MetsBlog.com), and Mark Simon (ESPN) will be discussing all things baseball in NY. Moderated by Carson Cistulli.

Baseball Media (9:45am – 10:30am)

Jonah Keri (Bloomberg Sports) will host a panel comprised of Will Leitch (Deadspin, New York Magazine), Michael Silverman (Boston Herald), Matthew Cerrone (MetsBlog.com), Alex Speier (WEEI.com), and David Biderman (WSJ) to discuss how baseball media coverage has changed in recent years and will continue to evolve.

Baseball Stats (10:40am – 11:15am)

Jon Sciambi (ESPN), Mitchel Lichtman, Sky Kalkman (Beyond the Boxscore), Dave Cameron, and David Appelman will discuss where advanced baseball stats are right now and where they’ll be headed. Moderated by Carson Cistulli.

Bloomberg Sports Presentation (11:20 – 11:35)

Bloomberg Sports will make a presentation of a brand new product.

FanGraphs Q&A (11:40 – End)

Dave Cameron, Carson Cistulli, Bryan Smith, Joe Pawlikowski, Mike Axisa, and David Appelman will take questions until we’re officially kicked out (a little after 12:00).

Afterparty (3:30pm – Game Over)

Additionally, we’re going to host a game-watching party for attendees to gather at a local watering hole and view that afternoon’s Boston-New York match-up together. Those who make it to the event will be invited to join us for several more hours of fun later in the afternoon. Details and directions will be given at the event.

I found Tangotiger’s most recent post on what if every player in the majors leagues was exactly the same as something anyone who has the slightest interest in baseball analysis should read.

Suppose that God herself came to you and told you that she was going to do something devious: for the 2011 baseball season, every team would have 25 players of identical talent, with all 30 teams being equals.

If you tuned out for the weekend and missed any of our coverage of the trade deadline on Friday or Saturday, here’s links to our analysis of what went down:

Marlins Acquire Ohman From O’s For VandenHurk

Royals Trade Away Ankiel, Farnsworth

The Giants Add Pen Arms

New York Yankees Acquire Kerry Wood

Dotel to the Dodgers

Ted Lilly and Ryan Theriot to the Dodgers

Cardinals Add Jake Westbrook, Lose Ryan Ludwick

Pirates Lift Chris Snyder

The Ryan Ludwick Trade: San Diego’s Perspective

New York Yankees Acquire Austin Kearns

Rays Acquire Chad Qualls

The Lance Berkman Trade: Yankees Perspective

Cristian Guzman to the Rangers

The Edwin Jackson Trade: Arizona’s Perspective

The Edwin Jackson Trade: Chicago’s Perspective

The Miguel Tejada Trade

New Prospects for Nats, Jays, O’s

Baseball Info Solutions has just released a more comprehensive FAQ on their fielding system, which we list on FanGraphs as DRS (and the various components that make up DRS).

It goes into details about how they make adjustments for various positions, ball hogging, home runs saved, the Green Monster, player positioning, etc….

Click to read the FAQ