The Linearity of Cost per Win
This is Matt Swartz’ third piece as part of his July residency at FanGraphs. A former contributor to FanGraphs and the Hardball Times — and current contributor to MLB Trade Rumors — Swartz also works as consultant to a Major League team. You can find him on Twitter here. Read the work of all our residents here.
In this series of articles, I analyze the average cost per WAR on the free-agent market, as well as looking back at previously discovered market inefficiencies to see how they have changed over time. However, in doing this analysis, it is important to ensure that any assumptions I make have theoretical and empirical backing, including perhaps the largest such assumption — namely, the linearity of the Cost per WAR on the free-agent market. Does a four-win player earn twice as much as a two-win one? Some analysts have argued that, due to scarcity, a 4-WAR player could earn more than twice as much, although I have shown in the past why I believe this is not likely. Today, I will confirm linearity is still a fair assumption to make.
First, it’s useful to discuss the economic implications in theory. The question of linearity comes down to how easy it is to replace a four-win player on the free-agent market, and if teams would be better off going after two 2-WAR players. If so, teams would drive up the price of 2-WAR players and drive down the price of 4-WAR players as they got smarter over time, until both methods of acquiring 4 WAR cost the same. However, perhaps teams cannot upgrade at any enough positions to enable this kind of arbitrage. As revealed by analysis I’ve performed in the past, there are, in practice, many different options a teams has. Nearly every team has a lineup spot, a rotation spot, and a couple of bullpen spots open in any given offseason. Many have more, and teams also have the option of conducting trades, as well, to make room for upgrades if so desired.
None of this says that some teams would never choose to take the approach of going after more 2-WAR players in lieu of going after big names. Individual teams are bound to have different assessments of replacement level both for their own team and the market in general. A team that felt that they had a high replacement level internally would be more inclined to go after big-name players and fill in the remaining spots with their internal high-replacement-level players. Alternatively, a team that felt replacement level was much lower than the market suggests would spread their spending across multiple players to avoid having to fill a vacancy with such a poor player.
As mentioned, my previous findings suggested that Dollars per WAR was linear. To see if this is still true, I split the market into three periods — 2006-09, 2010-13, and 2014-16 — and looked at the cost per WAR using my framework discussed in the previous article in different ranges of salaries (net of the league minimum). This does lead to some sample-size issues, but here is the relevant table:
| Net AAV Range | 2006-09 | 2010-13 | 2014-16 |
|---|---|---|---|
| $0-2 million | $3.3 | $2.7 | $26.5 |
| $2-5 million | $5.3 | $5.7 | $13.1 |
| $5-10 million | $5.9 | $5.7 | $7.5 |
| $10-15 million | $5.4 | $7.6 | $7.2 |
| $15-20 million | $5.6 | $7.6 | $11.6 |
| $20+ million | $4.9 | $7.4 | $10.3 |
| Overall | $5.4 | $6.5 | $9.0 |
And here’s that data rendered into visual form:
As you can see, the dollar amounts per win retain a general proximity to the overall averages for each time period. Early numbers did show some non-linearity in the very low-$ part of the market (under $2 million net AAV) but that was probably related to measurement error. Such deals are often one-year deals with sizable incentives that are poorly reported. They also overwhelmingly go to players just above 0 WAR, and therefore are highly vulnerable to measurement error of WAR itself if replacement level isn’t measured correctly. A slightly higher approximation of replacement level could lead to a much higher $/WAR estimate in this range.
I probably was less likely to miss out on incentives in more recent deals when collecting data, and there is actually a large over-correction where $/WAR is very high in the lowest salary bucket for 2014-16. Overall, I think it is best to focus on deals more than $2 million above the league minimum. You will see that the above issue led me to focus only on deals in excess of the amount for much of the subsequent analyses.
But once we get past that first row, we can see strong evidence of linearity in all ranges. The most recent years (2014-16) do show a little bit higher cost per WAR in the high-salary ranges, but since they also do in the low-salary ranges, I suspect this is just noise, and I am comfortable using a linear framework to Dollars per WAR in subsequent articles. This jump in $/WAR at high-$ levels (in the last column) is probably also a function of the small sample sizes as well. There are just 80 and 74 player-seasons respectively in the top two salary groupings for 2014-16.
Any non-linearity in cost per WAR would severely complicate the analysis of the free-agent market. I would certainly welcome this complexity if it were warranted, but I think the evidence and theory both clearly point to linearity making far more sense.
In my next article, I will explain the calculation of draft-pick cost in the Dollars per WAR framework, and the importance of discount rate while doing so. Once that piece is finished, the framework will be defined clearly enough that we can begin looking at the evolution of market inefficiencies.
Matt writes for FanGraphs and The Hardball Times, and models arbitration salaries for MLB Trade Rumors. Follow him on Twitter @Matt_Swa.
Wait a minute… If a win is worth $10M(ish), and we know that there are players who can reasonably be projected to be worth more than 4 wins per season (and sometimes more), why is the top salary in baseball (Kershaw) only $35M? According to FG, there were 45 hitters and 15 pitchers who were worth more than 4 wins last year.
Maybe there’s a case to be made that teams are cost-shifting some of by adding on more years to a deal (and that teams are pricing for back-end risk), but even after adjusting for that, the top of the market still seems to be very distorted.
This is a very common question, and there are two main reasons why:
(1) Prospective projections rarely forecast players to be worth much more than 5 wins per year
(2) Teams almost NEVER sign players to deals where they are forecast to be worth anywhere near 4 WAR in their final year. Usually multi-year deal are signed such that the player averages maybe max 3 WAR overall across there deal (e.g. starting at 5 WAR and ending at 1 WAR), so a maximum near $30MM makes sense.
Exactly, exactly and exactly. Russell I can’t believe that YOU would say this, “According to FG, there were 45 hitters and 15 pitchers who were worth more than 4 wins last year,” when surely you know that salaries are based on projected wins and surely you know that the highest wins in retrosepct will always be much higher than their projections.
I get that, I’m making a point about the fact that players who would “break” the line are not all over the place, but they aren’t uncommon.
Even ignoring the projection/performance gap, the deals are almost always signed with an idea that the player is not a star at the end.
But the data don’t tell that pessimistic of a story. I did this one with position players (need to do this for pitchers still), but I took players who were top 5 at their position in WARP (I used BP for this one) for three years straight. Three years later, 38.4 percent were still top 5 and 33.9 percent were still first division (ranked 6-15). Five years later, those numbers were 17.0 percent and 29.5 percent respectively. If you sign a super-star, even after a few years, you’re still likely to end up with a “pretty good” player, the kind of guy who would still be worth at least 2-3 wins.
Maybe the lesson here is that teams over-fear downside risk?
How many of those guys already had six years of service time already? I’m also not sure that telling me 70.5% of superstars are going to below average players five years later makes your point very well here either, even if we ignore the service time-age selection bias.
I don’t think teams over-fear downside risk because they seem to be paying $/WAR in line with the $10ish in practice.
Russell, you’re trying to tell me these guys in practice get low $/WAR, and I have data above that shows that isn’t true. Look at the “$20+ million” row– it’s roughly in line with the “overall” $/WAR in practice.
Because of the way that data were set up, I had to limit by age, but I recall that I did a sub-analysis of age 27-32 (the kinds of guys who are getting mega-deals) and the numbers still came out favorably. (I’m on the wrong computer, so I don’t have that data file on-hand to re-run.)
I mean, it’s still a proxy analysis for the actual here, right? It’s not looking at the $/WAR of the high end of the market. You’re attempting to argue that the top end of the market gets low $/WAR, but in practice doesn’t the above show that your claim is wrong?
Something looks off to me in that $20M row. Assuming those are deals that average out to being say $25 million per year, and teams are paying $10.3 million per WAR, that suggests that teams believe that over the course of the contract (which is probably longer than the average deal) they’d be getting 2.5 WAR per season. Sure, often the entire point is that some of that value is front-loaded, but the kind of guy who gets that sort of deal is probably at least realistic to project as a five win guy for the next year or two, and that “buys out” a completely worthless year or two on the back end. Those back-end dud years do happen, but the evidence suggests that star players tend to at least hold some of their value, and even if they’re reduced to 1 win players in year 8, that’s still 10M of value.
The data is not projections. It’s actual WAR of those deals. When I say $10.3 MM/WAR for $20+MM for 2014-16, I am talking about the actual 74 player-seasons that had $20+MM AAV over the previous three years. All 74, including the guys who had deals that had deals for 2016-23 and the guys who had deals from 2008-14.
A note on the figures. I’m saying that 17 percent were still top 5 after 5 years, and 29.5 percent were ranked 6-15. The percentage who end up as below average regulars is 53.5… roughly half. I had another 14.3 percent that are still second-division starters, meaning we’ve got a roughly 2/3rd chance of producing some value over replacement-level, even after five years.
So…the median is about 1.5-2 WAR? Even before accounting for the older subset of players?
I didn’t run it like that at the time, so I’d have to look. But that’s probably about right.
Seems like they’re going to average right around where I thought they would, around 3 WAR full-contract projection on average for the very top guys.
When I say 1.5-2 WAR, I mean that’s what we can expect out of them (median) in year 5. If the AAV is $25 million, then that’s not full value, but it’s close, and years 1-4 were probably a lot better.
Probably not much better in year 4 than year 5. I think the average would be a gradual decline from about 4.5 to 1.5 in your sample on average, or something like that.
In your opinion does point (2) represent a market inefficientancy on they player’s part? Or is this inefficiency caused by owners being inefficient such that a owner would NEVER pay $50 million a year to a guy that on average, according to the models, is worth $70 million a year in production?
I don’t think it’s a market inefficiency because I think most players (or at least their agents) know that they are likely to decline. They are better off locking down a multi-year deal with AAV based on their average expected production rather than going year-to-year and having wild swings in salary.
I get that guys who legitimately project to be 4-5 (or more) win players aren’t everywhere, but there are a few of them. Rationally, we can say that four years from now, we have to build a lot of uncertainty into any projection and that’s going to bring the expected value down, although “rationally” doesn’t always describe humans. (Insert lecture on Optimism Bias.) If nothing else, the teams that are giving out these contracts are usually on the sweet part of the win curve, so they would be incentivized to be a little looser with their pocketbooks.
Teams generally solve this problem with cost-shifting through extended terms (although I looked at that and found that on average, FA contracts in the top 10 (by AAV) over the past few years got something like 1.5 more years than 11-20 and 21-30. That is extra risk, but the guys who are generally getting these contracts are the superstars who even if they tumble, are tumbling from a starting point of 6-7 WAR.
We’re running into an orders of magnitude problem. This would be less of a problem if we weren’t simultaneously saying that the slope of the line is “$10M/WAR.” Now we’re talking about any player whom we could reasonably project to average 3 WAR or so over the life of a contract to “deserve” a salary of $30M. There are a lot more guys who fall into the previous category than there are guys who are getting paid $30M. Heck, in this scenario, a 2-WAR guy should be getting $20ish million per year, and that guy sure isn’t getting a 6-year deal.
Maybe the linearity assumption holds in the middle of the market, and perhaps that’s the majority of cases, but there’s a pressure cooker building up steam here, with real consequences. If I’m going to have to pay a 3-win guy $30M, and I can get Kershaw for $35M, even building in the extended term and back-end risk, buying at the top end of the market seems like a steal from a value standpoint. If the market is linear, then something has to give. Either the next Kershaw (Machado? Harper? Trout?) will need to get something more like $50-60 million per year or alternately, the price in the middle of the market will shift downward. The twin statements of linearity and “$10M per win” mean that uncomfortable middle is growing. Maybe there’s a shock to the system in the next year or so that makes the market more linear.
The alternative is that the market just isn’t linear at the top. I think that there are some interesting reasons why that might be and it has implications for strategy.
Much of my counterpoint is that if you scroll up, the market IS actually pretty linear, including at the high end of the market.
You can tell me people aren’t rational as economic models, and I’ll tell you that I agree and point you to my author page to show you some of the examples I’ve found. But linearity is rational and it is present in the market, even at the top.
Free agent spending overall is biased towards teams at the peak of their win curve, so I’m not sure I follow that logic. Even when it isn’t, most free agents at least get some offers from teams near the peak of their win curve.
How long were the deals at the 6-10 range by the “top AAV”? There are rarely very long deals in that range anyway. What would happen if you did top 2 or 3?
Can you think of many recent free agents who signed long deals and were projected for much more than say 2 WAR at the end? Guys who reach FA projected to get 6 WAR tend to sign 8-10 year deals. I can’t think of a long contract like this where people thought he was underpaid using this framework. Your point hinges on guys signing deals where the consensus was they would be worth 4-5 WAR at the end, and I can’t think of anyone like that (let alone several or most), and I’d be skeptical anyone you mention ends up there either.
I did find some data on my data stick that can shed some light here. I have the top 30 FA signings in each off-season by total contract value, back to 2011. I paneled it by your AAV groupings that you used in the article. The short version is that players who got $20M+ in AAV got an average 6 year term. Everyone else was checking in around 3. My dataset doesn’t include the low-end $1 signings for the most part, but it’s realistic to think that a top-end guy is getting about 6 years.
Yoenis Cespedes notwithstanding, they were selected for those deals based on the idea that they were at least good for superstar performance in the near future. At a salary around $25M, we’re again expecting roughly 2.5 wins from them on average over the life of their contract, which is about what we expect from a first-division starter. If they are realistic to project to twice that in their first couple years, and history tells us that they have a half-and-half chance of still being a first division starter or better in five years, it seems like an underpay.
Okay…it “seems” that way based on a back of the envelope calculation. But scroll up: above is the whole Postal Service– it wasn’t actually an underpay!
I get that the numbers line up, but the “why” is important.
What I’m saying is that there is an identifiable type of player that busts these calculations (and by a not-insignificant amount), even once you control for that back-end risk. He’s young (in free agent terms) and he’s really good. Those guys aren’t all crawling all over the place, but they’re around. Perhaps I’m just being too bullish in how I’m projecting the risk, or perhaps the current projection models are too bearish. That could be worth a look. Maybe the issue is binning solely by AAV on FA contracts. That tosses in the guys whom we’d probably consider model breakers with deals that fall in the “eh, that’s about right” category and the Cespedes deals of the world.
Let me instead frame things this way. In 2007, the highest paid player in baseball was Jason Giambi at $23.4 million. This year it’s Kershaw at $35.6. The ratio of Kershaw to Giambi is roughly 1.5:1. Your numbers from last week had $/WAR at $5.3 in 2007 and $10.5 this year, almost exactly a 2:1 ratio. In theory, the best paid player in baseball should be the best player, if the market was rational. That of course, does not happen, but the team that signs that sort of deal thinks that the player whom they are signing has that sort of potential. We can look at the top salary (or top set of salaries) and see what the league thinks that “the best player in baseball” is worth. I find it interesting that the two numbers haven’t grown at the same rate.
Here is my alternate formulation. The market is linear and rational up to a point when we’re getting into contracts that aren’t about living expenses any more for the player. Maybe I could make a case that Clayton Kershaw is worth $50 million or so per year, but marginal utility-wise to him, what’s the point? The higher up you get, the more we’re then in the land where the size of your contract is a mark of social stature, and this is where very non-linear and very not-rational things take over.
If I were to go to my GM and demand a contract of $40 million per year, someone would probably ask, “Is he better than Kershaw? Is he deserving of being the best paid player in baseball?” rather than “What is the rest of the market getting and how does that pro-rate to his projection?” The first questions answer where I fit in the social pecking order. The second question is linear.
So your contention is that a subset of the $20MM+ bin is predictably underpaid and another subset of the $20MM+ bin is predictably overpaid. It’s possible. But I’d be very surprised if it broke down neatly by age like that, and you’d need to prove it anyhow. A sample size of two (Kershaw/Giambi) is not very persuasive. Especially since Kershaw’s deal was signed as part of an extension, which I’ve already shown may times before are already $/WAR underpays on average regardless of AAV.
The length of deals in practice has a lot more to do with the luxury tax and the draft pick compensation rules than you might think.
Another huge issue that might seem nitpicky but really isn’t is your claim that the best player should line up with the highest salary if things were rational. That is not true because most WAR goes to players who have less than six years of service time. That wouldn’t matter as much except for that fact that the pool of >=6 service time players has shrunk by about 30% over that time period, making the odds the best player falls in there fall too.
It’s entirely possible to identify subsets of players who the $/WAR is persistently below the market average. Most of this series of articles will actually talk about those. And that can be evidence of market irrationality or other factors, which I’ll explain later. But the burden of proof on “100% of the teams full of people who do this for a living are missing a big opportunity” is on you, just as it’s been on me when I’ve done this before, and back-of-the-envelope calculations aren’t proof of fact. I don’t think this is going to be one of these examples because I think it’s too sexy of a group for it not to push teams the otherwise. For eight years, I’ve been trying to argue with people that the group you’re talking about isn’t overpaid. “The baseball FA market doesn’t like young superstars enough” is not a common refrain I’ve heard.
There should be some kind of curve at the top to reflect risk aversion, right? I mean, if a player was in theory worth $100M/year, he wouldn’t be worth that in practice because if he got hurt, you’d have zero chance. Perhaps no real players are worth enough to make that a factor, but I imagine most teams would prefer diversification over concentration in their WAR portfolio. I.e., if Kershaw was twice as good in terms of expected WAR, he would (rightly) not get paid twice as much.
Why should baseball team want a lower risk team? If you expect to win 85 games, would you rather be stuck between 82-88 or 77-93? The latter is sometimes a playoff team and makes way more money on average. Remember the $100MM/year value isn’t a “most likely” but a “mean”. It would mean there’s probably $150MM sometimes but sometimes he gets hurt too.
Because if you are a rich team, you have an edge. Adding extreme volatility [a $100M/yr player] will reduce that edge. There is no advantage to making the playoffs with a 20-game lead. Put another way, if I’m in a fantasy league and I have twice the auction salary cap that the rest of the league has, there is no way I should put all my eggs in one basket. Rather, I should be building a deep, safe team.
Also, regardless of what is better strategically, in practice, sports teams always try to reduce risk. No one wants to go 0-16 in NFL or 40-122 in MLB with regularity. Why? Because everyone but the owner will get fired if that happens with a big-market team. At the end of the day, people want to keep their jobs — which is why you often see NFL coaches playing to make it closer when trailing late rather than taking extreme risk to pull off an unlikely win.
Even with a balanced team, natural variance for expected win totals is far greater than the “82-88” (+/- 3) win example you give. The game itself has plenty of volatility built in. Massively increasing it further would serve no useful purpose for the majority of teams.
The natural variance was for illustrative purposes only, but you are correct that the wins will be distributed roughly binomially. But check how few teams start the year with >50% playoff odds. Since expected win totals are close to binomial, and even more likely unimodal and symmetric, it will be rare that teams will not see higher playoff odds from variance increasing conditional on expected value.
You missed the two biggest points here.
1. Due to the popularity of extensions, fewer players who would be surefire 4-win projections are making it to the open FA market. Neither Cespedes nor Justin Turner really hit that mark last year, and all of the big names of the upcoming class are all having mediocre-to-bad seasons. Extensions usually involve buying off arbitration time, so guys like Stanton, Kershaw, Trout have just started the payout earlier, at a cost to AAV.
2. The guys who have hit open market at a skill level to justify $40MM+ compensation (Price, Heyward) have all taken opt-outs or no-trades, which dinged their AAV.
Now, I do think this is partly because teams just can’t stomach paying more than $40MM to one player, and having that on their books. If any GM ever did give out that kind of contract to have it flop, they’d be the ultimate laughingstock. Even though it doesn’t really reduce price, opt-outs, extensions, clauses, and incentives all keep teams from having to put “$45 MM player ” on the DL for a season. That is not not valuable.
It will be interesting to see if this effect comes into play when Harper hits the FA market. That relatively low max salary is only possible because agents and players play ball on financial maneuvering. Much like A-Rod was hilariously underpaid for his first FA contract in Texas, if Harper has the personality to insist on the biggest possible yearly payout, he could end up being the biggest FA opportunity of the season.
If he’s a six-win player at $12 MM per win, he should fairly make more than double the current yearly salary. If Harper refuses salary diffusion through opt-outs, will some team really shell out $50MM a year for 10 years? It seems like they’d be fools not to.
There is more than just a case on cost shifting as long contracts are generally structured such that salary is generally constant or backloaded while the player’s talent typically decreases the further he is from his prime.
I imagine if Kershaw was going year to year he’d be getting $50-60M each year.
I wonder if anyone is going to do that in the near future. Harper could assuredly get 2 years 100 million. That might be on the low-end of what he would get if he only wanted a 2 year contract. I coud see someone like the Yankees giving him an aav of 60-70 million for a two year contract. I understand that contracts tend to be for longer time periods to help mitigate the risk of injury to the player, but if anyone would bet on themselves it would be Harper and Boras.
Doubtful.