A Retrospective Look at the Price of a Win

Barry Zito got paid $18.5 million last year and produced -0.4 Wins Above Replacement. You and I checked in at about 0 WAR — and by staying off the field, we out-produced Zito. I don’t know about you, but my paycheck didn’t look anything like Zito’s last year. And while my employer might be happy about that, the Giants are wondering what they got for their millions. In the everyday business world, an employer could find a way to get rid of a guy like Zito — an under-performing, overpaid employee. The Giants can’t. In fact, to get another team to take him, they’d need to pay most — if not all — of his salary. Regardless of his performance, Zito’s getting his money.

Then there’s Albert Pujols. He was paid $16 million last year and he produced 5.1 WAR. There’s nothing wrong with $16 million, if you were the St. Louis Cardinals. In fact, Pujols was worth a lot more. Teams fell over themselves trying to give Pujols a 50% raise this off-season, and that was no surprise.

So we know that the Giants had a bad deal with Zito, and the Cardinals had a good deal with Pujols. Those are extremes, and they’re obvious. But baseball, like life, is usually lived in a gray area — like the players who got the medium deals. But to figure out who those guys are, a more sophisticated analysis is required.

In today’s and tomorrow’s articles, I’ll determine the average price paid for a win in the past five years. To do this, I’ll first explain the difference between the expected price of wins before a season begins and the actual price paid after a season is finished. There are several important differences between expected price and actual price, and they’ll matter when we take a retrospective look at these contracts.

Most businesses try to get the most bang for their buck on labor, and baseball’s no different. Sabermetricians for years have been figuring the price of a win — which helps them sort the good deals from the bad. FanGraphs’ “Dollars” metric estimates the value that teams place on production in the free-agent market. The formula is straightforward: Take the average annual value of contracts given out to free agents, then use Marcel projections to match the contract values with the players’ expected production.

The FanGraph’s method is basic and gives us apples-to-apples comparisons of contracts going into the season. Naturally, by using the total salary paid to free agents — divided by their total expected WAR — roughly half the contracts rate above average, while the other half rate below. Simple, right? Maybe not: When we look at past season, things don’t exactly add up.

Take the past six seasons as an example. From 2007 to 2011, there were 1,436 players with at least six years of service time. If we take the retrospective value of those contracts using “Dollars” and the actual salaries paid for all of those guys put together, we might expect these two to be equivalent. The “Dollars” statistic is supposed to represent the average expenditure per win on players eligible for free agency. But look what happened:

“Dollars” Value: $7.41 billion
Actual Salaries: $8.46 billion

This means that players get paid more, on average, than we expected the average player to be paid. This happened because players under-produced their expected WAR by 16%. That doesn’t mean using projections to develop a market value is useless, but it does mean that evaluating deals in retrospect requires some sort of adjustment.

Why did these players under-produce relative to expectations? There are a few reasons:

1) Playing time projections for free agents are too high

Ask anyone who developed a top projection systems about playing time, and they’ll tell you that’s not what they do. They only attempt to estimate rate statistics, but they do a pretty good job. Still, they generally come in way too high on playing time. Taking all 797 players and 638 pitchers with six years of service time or more, I checked all the Marcel, Oliver and ZiPS projections that I could find, and I discovered a common trend:

	N	PA	AVG	OBP	SLG
Marcel	782	461	.272	.344	.438
Actual	782	384	.269	.339	.424

 

	N	PA	AVG	OBP	SLG
Oliver	755	491	.271	.339	.433
Actual	755	391	.269	.339	.424

 

	N	PA	AVG	OBP	SLG
ZiPS	776	441	.271	.344	.432
Actual	776	384	.269	.339	.424

The three rate statistics for batters were projected pretty well. They came in a little high, but that was probably due to the recently declining run environment in baseball. The playing time was over-projected by about 20%, which means that WAR was over-projected for hitters.

The same result held for pitchers:

	N	IP	ERA
Marcel	611	103	4.25
Actual	611	91	4.20

 

	N	IP	ERA
Oliver	541	113	4.16
Actual	541	94	4.28

 

	N	IP	ERA
ZiPS	596	110	4.24
Actual	596	93	4.19

Pitchers actually had better ERAs than their projections suggested, but this was also probably due to the decreasing run environment. But much like the hitters, pitchers played less than expected. In fact, they pitched about 17% fewer innings than the estimations.

None of this is an indictment of projection systems. But the result of over-projecting playing time means an over-projection of WAR. The end result of that means under projection of the $/WAR paid on the free-agent market.

Over-projecting playing time isn’t entirely unique to free agents. In last week’s article on testing projection systems, I evaluated how close projections were to wOBA and ERA, which are rate stats. What I didn’t show is playing-time statistics. Using all systems for only 2011, we see that over-projecting playing time was common, but it wasn’t an epidemic.

Projection	PA Over-projection	IP Over-projection
PECOTA	18%	5%
ZiPS	15%	5%
Oliver	18%	8%
Marcel	-2%	-11%
Cairo	13%	-3%
Steamer	-2%	-6%

The plate-appearance over-projection is even more surprising, because I only included players who got at least 200 PAs. The most egregious over-projections are excluded from this. And even though only some projection systems look high for pitchers, the 40-innings minimum also excluded the worst over-projections. These numbers are all somewhat under-estimated.

Notice that not all projection systems over-estimate playing time. In fact, when looking at projections across the board, Marcel doesn’t over-project playing time, but it does over-project playing time for players who are eligible for free agency. Interestingly, the systems seemingly under-project playing time for players who aren’t eligible for free agency.

The main lesson to take away from this is that playing time matters when we’re evaluating free agents. If we’re only looking to evaluate deals relative to each other, we can get away with over-projecting playing time across the board. But if we want to make a statement about what Albert Pujols does for the Angels, we need to take an additional step and ask whether we are over-estimating his WAR, even if we correctly estimate his wOBA.

2) Players decline throughout contracts, making $/WAR lowest in the first year

To estimate the value of free-agent contracts, we generally look at just at a contract’s first year, because contracts signed several years ago have little to do with the current market. Unsurprisingly, players decline over time. As a result, the $/WAR in a deal’s first year is generally higher than the $/WAR in later years.

In fact, if you look at the $/WAR paid for players with at least six years of service time throughout the 2007 to 2011 seasons, you’d get the following ratios:

First year of contracts: $4.25 million/WAR
Later years of contracts: $5.5 million/WAR
Combining all contracts: $4.92 million/WAR

Players in the first year of their contracts get paid almost $1 million less per WAR than all free-agency eligible players, and that causes a retrospective over-estimates of $/WAR. Contract evaluation needs to take into account the fact that the most recent contracts are the most relevant, and the fact that the first year is usually the one that gives the best bang for a team’s buck.

3) Free agents who sign elsewhere generally give their new teams less bang for their buck.

This was detailed in my recent article in The 2012 Hardball Times Annual, but a look at a couple results from that piece will tell an important story here.

The $/rWAR* for all 196 multi-year deals that ended in between 2007 to 2011:

Re-signed players: $4.9 million/WAR
Players who switched teams: $6.8 million/WAR
Difference: 39%

Looking only at deals between two and four years long, I found the following differences:

Re-signed players who signed before the end of the season: $4.6 million/rWAR*
Re-signed players who signed after the end of the season: $6.6 million/rWAR*
Players who switched teams after the season ended: $8.9 million/rWAR*

*Note that rWAR is different (and usually lower — than WAR used at FanGraphs.

There’s a clear bias if we look only at the contracts for players who reached free agency; these players are more likely to be overpaid, which also throws more measurement error into the data.

4) The amount surrendered for a free agent includes both the dollars and the draft picks that are given up.

In fact, when I say that “Dollars” is limited in retrospective because of the under-production of free agents, I’m not even taking into account an additional cost that teams consider when signing free agents.

In this article, I showed how to estimate the costs of draft-pick compensation when signing free agents. Specifically, I explained that teams would pay 10% more for free agents overall if they didn’t also have to surrender draft picks. I also gave the following estimates of the dollar value that teams place on draft picks:

Surrendering 16th overall pick: $8.3 million
Surrendering 20th overall pick: $7.3 million
Surrendering 30th overall pick: $5.7 million
Surrendering 50th overall pick: $4.1 million
Surrendering 60th overall pick: $3.7 million
Surrendering 99th overall pick: $2.6 million

Re-signing Type A Free Agents: $9.6 million
Re-signing Type B Free Agents: $4.8 million

Going Forward

Coming up with a well-reasoned retrospective estimate of the average dollars paid per WAR for free agents is tricky. Obviously, we now know that playing-time projections are too high, but dealing with “Other People’s Player Bias” and early extension bias is something that we’ll need to understand better.

If you come away from this piece with more questions than answers, you’re reading correctly. In the next few days, I’ll be writing more articles that explain how the free-agent market has behaved in the past five years. I’ll also show some interesting idiosyncrasies about how teams pay players. After we’ve established how the market has worked in the past, we’ll have a better sense of how it might function in the future and how teams could better approach free agency in the future.