Building a Better WAR Metric
Wins Above Replacement (WAR) has as its genesis Bill James, even if Bill might not necessarily take the credit (or blame based on some readers) for it. But make no mistake, Bill provided the plumbing for it. For those interested, you can read Brandon Heipp’s account on that backstory.
When you put all the plumbing together, you can create a framework. And that’s what WAR is, a framework to provide an estimate. Wins Above Replacement is an estimate of… something. What that something is is different for every person. While the currency is wins, it’s not clear what those wins represent. There are reasonable choices you can make along the way. And for every fork in the road you take, you may diverge yourself from the next guy. This is why WAR can never be one thing.
As a framework, WAR leaves little room for discussion. Whether it’s what you see at Baseball Reference or at FanGraphs or openWAR or (to some extent) at Baseball Prospectus, they have as their framework the WAR that was championed on my old blog, which culminated with this article. But a framework is not the same as an implementation. 95% of the cars on the road all follow the same core design. That’s the framework. But a Chevy is different from a Lexus. Those are implementations. And there are as many implementations of WAR as there are baseball fans. Whereas Baseball Reference and FanGraphs and the others provide a consistent, systematic implementation, most fans have their own personal mish-mash of arbitrary, biased, and capricious combination of stats, which can change as their mood fits.
This series of articles, of which there may be a dozen(*) is an effort to try to come up with a WAR metric that will satisfy the Straight Arrow readers.
(*) I have no idea. This is the first one I’ve written.
***
I’ll ask you a series of questions, starting now. The openWAR guys talk about “preservation of runs”. That is a good starting point, and a great way to describe the concept. So, the question centers around whether we want to make sure that everything adds up at the play level. If you get a bases-loaded walk, do we want to make sure that exactly one run is accounted for or not?
If you care about “talent”, you just want to account for around +.30 runs for offense (and -.30 runs for defense), because you don’t want to be concerned with the specific base-out state. (We’ll talk about “preservation of wins” in a later question.) Similarly, is a bases-empty walk and bases-empty single the same thing or not? And if you want to preserve runs, are you ready to accept a bases-loaded walk and a solo HR as being the exact same thing?
So, have a discussion, and then answer this poll question:
There are plenty of other discussion points that go into building an implementation of WAR, and we’ll get to those in the future. For this post, I’m interested to hear what you guys think about this issue specifically.
Thank you guys, you have spoken loud and clear. I’ll post something new up tomorrow.
I voted with the majority here, however my real position lies somewhere in between. Pitchers approach hitters in different ways depending on the game situation, and ideally that should be taken account of. A walk with the bases loaded is certainly not the same thing as a solo home run from the perspective of the batter because in one case the batter fully accounts for the run scored, while in the others it is quite a team effort (of course from the team perspective the walk with the bases loaded is better than the solo home run because you are in better position to add more runs).
Perhaps a more interesting question is whether a two out bases loaded walk is the same as a two out bases empty walk? Here you have the same outcome, but the pitcher is approaching the two situations very differently because the likely scoring outcomes are very different. I think it is these types of situations where a walk can be weighted a bit more heavily for a hitter.
But the issue then becomes did the hitter do anything? A walk does involve the hitter not making an out, but it can also be a hitter just standing there and letting the pitcher dig his own grave. Putting a different weight on the outcome for the hitter when the pitcher is the potentially the only one making a change in approach seems a bit odd. If the objective is to determine what the hitter does to be successful, then it would try to determine the hitter’s action, not how a pitcher adjusts to the situation.
It’s a problem where trying to dig too far into context leads to difficulties in not only trying to weight the overall action appropriate, but also trying to determine who did what and to the extent they did it. While possible to determine it, there is also the issue of whether the search for that information results in anything substantially better than a “less precise” metric.
I could have made it even starker, by comparing a solo HR to a bases loaded Hit By Pitch.
The objective is to try to find how people think. And by asking a series of black and white questions (or as best as I can come up with them), we can infer what motivates the fan, or at least the fans at Fangraphs.
Thanks for participating.
I’ll be asking a dozen questions. And that one, or one like it, will be on the docket.
Feel free to post other questions you’d like me to consider, or email me at:
tom~tangotiger~net
Great point by London Yank. I went the opposite way (in favor of preservation of runs) only because there may be skill involved in adjusting your approach to the base-out situation, and I lean towards seeing that reflected.
(Truth is, I’d love to have stats done on each basis. Glad they DO exist.)
I’m not the best at reading between the lines but is this basically asking if things should be context driven or not? It seems like context driven wouldn’t be any more descriptive than regular stats except that you could compare players slightly easier, yet heavily discount players on bad teams. “Traditional” WAR is context neutral because base-out states are outside of player control so it gives everything in a neutral state. I could see the other having use only in fantasy baseball, and even then only a little
I wouldn’t try to read between the lines. I’m presenting specific situations with the goal of understanding how fans respond.
I’ll ask a series of such questions, and based on the responses we can infer how fans think.
It’s tempting to boil this down to a question of how much context should drive things, but I’m glad Tangotiger phrased the question in such a well-defined way. I don’t think preferring HRs to weigh more is preferring context. Instead, I would hope it reflects my preference for weighted statistics, i.e., I don’t care about a specific context, but I do care about how many runs a HBP generates, on average.
I want a third option. It depends on how many pitches my home run hitter saw. In general, I’m on the context neutral side- a home run is usually better than a walk. But I think there’s a lot of room in context neutral to use pitchFX and similar tools to add depth to context neutral.
I know it feels that a 10 pitch strikeout can swing the game as much as a first-pitch single. I wonder if it’s true.
Assume “all other things equal”. So, in both cases, the batters had a 3-0 count when the last pitch was thrown.
Hey Tom, I tried to get involved at your blog but I’m still waiting for my account to be activated.
I am interested in how the results of these questions will be used. Is it set to just be a talking point, or are you and other’s planning on possibly changing your WAR calculations. I think at the very least openWAR, who claims to be the WAR of the people, for the people, should reevaluate some of their major WAR building blocks to reflect how the majority of the people think WAR should be calculated.
1. I have now activated you on my blog, thanks for asking.
2. I really don’t have plans yet. At this point, it’s just to gather information. The openWAR guys for example make it a feature of their WAR that it has a “preservation of runs”. And, it is a feature… if that’s what you want. But, 20% of Fangraphs readers think that’s a feature, and 80% think it’s a bug. Or if not a bug, a feature they can do without.
But given that it’s a core design decision, openWAR will relegate itself to a niche implementation. There’s nothing wrong with that.
I’m just at the point of gathering information as to how people think and behave.
However, as these questions continue, we may even find that answers will have Orwellian logic. We’ll get to a point, maybe, that a is preferred to b preferred to c preferred to a.
At which point we can say readers aren’t consistent, so the WAR creators will have free reign.
“But, 20% of Fangraphs readers think that’s a feature, and 80% think it’s a bug. Or if not a bug, a feature they can do without.”
This is basically why I ask. The beginning of the openWAR paper had me really excited that there would be a version of WAR that any member of the public could access and use. My mood quickly changed as I read more about how they engineered their variation to be different than existing variations. I guess I’m not alone if 80% of fans disagree with a core design decision, and I have a feeling even fewer will agree with how they assign relative defense values.
So what is the point of having a version of WAR that meets the standard of reproducibility if no one wants to reproduce it. I’m more willing to use a implementation of WAR that was calculated in a “black box”, as they call it, because even if I can’t see the details, I agree with the primary assumptions used to build the model.
Well-said!
why is “alot” one word?
Pedantry aside; this is a fascinating question.
Base Ball used to be two words.
If you have more interest:
http://www.insidethebook.com/ee/index.php/site/comments/reader_mail_of_the_week_a_lot_v_alot/
I feel like WAR at its purest is a way to compare players down to a simple number. I fall more on the context independent side. For example, Josh Donaldson and Mike Trouts seasons were very similar, however Donaldson had many more RBI’s simply because he had more opportunities. If we want to compare these players they should not be judged on their team loading up the bases in front of them. I’ve played baseball, I know that a bases loaded walk takes discipline and shouldn’t be brushed aside but it cannot be anywhere near equal to completely creating your own run, as a home run will translate to any situation.
But the HR increases the inning’s run-expectancy than the bases loaded walk, and a leadoff out decreases RE more than a bases loaded out would (provided it wasn’t a DP).
Comparing a solo HR to a bases loaded BB: the change in run expectancy is identical.
(a) The expected runs before the event and after the event is unchanged, and
(b) The number of runs added to the bank is the same.