Determining the “Value” of “Valuable”

When the FanGraphs staff voted for the major postseason awards, I was the only guy who cast an MVP vote for Justin Upton.

I made my choice after calculating the Diamondbacks’ playoff probability with and without its 24-year-old star outfielder. With Upton, Arizona makes the playoffs 92% of the time. Without him, the figure drops to 30%. The success gap is impressive, and is the largest for any player in the National League.

And that’s why he’s the MVP.

I know that the world doesn’t need another “what does valuable mean” column, but I want to quantify a concept that has been around for years. Many writers and fans think that the MVP award should be something more than simply who’s the best player in a league. Among the most prominent, Grantland’s Bill Simmons, wrote about MVPs: “In my mind it means had this player not been involved, his successful team would not have achieved the same success.”

The NL WAR leaders in were Matt Kemp and Roy Halladay, who each had 8-plus wins above replacement. The reason neither of them got my vote is simple: The Dodgers weren’t going to make the playoffs with or without Kemp — and the Phillies would have won the division regardless of whether Halladay were making his living in Philadelphia. I’m not saying that these players were inconsequential to their teams, but they didn’t substantially swing their teams’ playoff chances.

Upton’s 6.5 WAR ranked seventh in the National League this season. But the crux of this theory is that wins are not created equal — and Upton’s 6-plus wins helped the Diamondbacks go from a team with little chance at the playoffs to one that would make the postseason 92 times out of 100.

If wins aren’t created equal, then which wins are worth what? Using regular-season results since the wild card was instituted in 1996, the probability that a team will make the playoffs once it reaches a certain win total looks like this:

The blue line shows the probability of a team making the playoffs with a certain win total. The reason it has this shape is that teams have almost no chance of making the playoffs with fewer than 81 wins — and teams will almost always make the playoffs with more than 98 wins.

The red bars show the marginal value of each win — or in other words — that win’s leverage. For example, a team going from 84 wins to 85 would gain roughly four percentage points of probability. This effect peaks at wins 89 and 90, which add about 12-percentage-points each.

To calculate the “value” of the player, you simply find the probability that his team’s record will put it in the postseason, then subtract that player’s WAR and calculate the probability again. The difference is the contribution of that player to his team. Upton’s looks like this:

No player moved his team further up the curve than Upton (+62 percentage points). Second most was Ryan Braun (+59) and third was Albert Pujols (+50).

I’m not suggesting that other MVP opinions are wrong. Those who buy into to the “best season” school of thought will probably vote for Kemp, Halladay or Braun. Historically, many voters subscribe to the “best player on the best team” theory, which would also benefit Halladay or Braun. Different criteria will result in different votes, and that’s fine.

Let me also say that this methodology relies heavily on WAR — so those who have issues with the statistic or question it’s accuracy likely won’t buy into my concept. I definitely think that there needs to be some common sense factored into what the numbers describe, and a few decimal points one way or another probably don’t amount to much. Even using this methodology, you could easily vote for Braun over Upton, considering there are only three percentage points between them (the gap was more than 10 points a week ago).

Although I fully support the idea that Upton should be the NL MVP, I wouldn’t use my methodology to fill out a top-to-bottom ballot. Kemp and Halladay would definitely get votes from me, even though they increased their team’s playoff odds very little (only +3 points for Kemp).

Perhaps the biggest critique I can foresee is the argument that this methodology is not specific to the 2011 season. After all, the D-backs did make the playoffs, so how can I say that their playoff probability is 93%? And even if you subtract Upton’s WAR from Arizona’s record, they still would have won NL West.

My counterargument is that each team can only control its own destiny, and a player’s MVP resume shouldn’t be rewarded or penalized for how other teams finish. The playoff probabilities are calculated using aggregate data from the past 15 years. In the same way that FIP is calculated using linear weights — which are the average value of a walk, strikeout and home run — I’m calculating linear weights for the average value of win number 85, 86, etc.

In the American League, things get really interesting. Using the same methodology, the top-five-most-valuable players are:

At the time of the FanGraphs staff vote, Ellsbury was the leader, and I gave him my vote. The final week of the season threw everything around, and now Ben Zobrist leads the rankings. Although Zobrist was super-valuable to the Rays, the top-five players are bunched so closely that there there is no clear-cut favorite. You could make a case for any of these five players and this methodology wouldn’t argue with you.

Debating postseason awards is an annual ritual, and one that I enjoy. My opinion isn’t intended to be an argument-stopper. If anything, I hope it adds another data point to argue about.