Do Teams Still Overpay for Free Agents from Other Teams?
This is Matt Swartz’ fifth piece as part of his July residency at FanGraphs. A former contributor to FanGraphs and the Hardball Times — and current contributor to MLB Trade Rumors — Swartz also works as consultant to a Major League team. You can find him on Twitter here. Read the work of all our residents here.
When I tell people about my side career as a baseball analyst, they frequently ask me of what research I’m most proud. The answer? The work I did establishing that teams receive fewer WAR per Dollar when signing free agents away from other teams than when re-signing their own players.
My clearest and most thorough analysis of this topic came in the 2012 Hardball Times Annual. The results were initially met with strong skepticism when I published a post on the topic at Baseball Prospectus back in 2010. It took a couple years of evidence before I was able to persuade the sabermetric community that it was true — and, more importantly, that the reason for this phenomenon was that teams re-signing their own players had better information on them.
My 2012 Hardball Times Annual article tested and confirmed that this held true for a variety of players. Traded MLB players and traded minor-league prospects both tended to underperform their projections when compared to untraded players.
What’s the significance of this discovery? Generally speaking, it means that an “average” player who reaches free agency is overvalued by his projections relative to another “average” player who doesn’t reach free agency. So much of sabermetric analysis involves looking at free agents. Suddenly, I had research indicating that such analysis was based on a biased sample. The results immediately colored every potential free-agent signing. With every free agent I encountered afterward, I began asking myself: “is there some reason this player’s original team let him go?”
When most sabermetric discoveries are made, they remain true even if the world knows about them. The physics behind batted-ball distance doesn’t change simply because we can more accurately measure launch angle or exit velocity. That pitchers exert more control over fielding-independent events (such as strikeouts and walks) than other outcomes remains true regardless of how commonly accepted that principle is.
That said, my “Other People’s Players” (OPP) finding is different (and shares something in common with catcher pitch-framing, too): the more widely understood the phenomenon, the less likely it is to remain true. Were teams to believe that free agents signed away from other clubs might be “lemons,” they would likely shy away from signing them.
This possibility is what I explore in the present article — and I’m actually pleased and surprised to find that the “OPP Premium” is eroding. Teams are becoming more careful when signing players from other teams and putting a premium back onto their own players when re-signing them.
Whether my analysis precipitated this development, I have no idea. I think it’s fair to suppose, however, that a general awareness of this phenomenon probably did drive teams to sign more of their own players — especially their own pitchers (on whom the OPP Premium was largest). But since many teams probably tested such things internally, I can’t verify that my own work was responsible for any of this. I will say that team officials with whom I’ve met with over the years have been aware of the concept, whether due to my research, word of mouth, or their own club’s confirmation of the same facts by way of internal research.
My initial analysis focused on multi-year deals because of the complications involved with analyzing one-year deals for free agents. Such players (i.e. those on one-year deals) were frequently close enough to replacement level that small changes in the approximation of replacement level would lead to very large changes in $/WAR (as the denominator changed drastically in percentage terms). One-year deals are frequently laden with incentives, as well, amplifying the effects of even small inaccuracies in the contract data I was able to procure. Throwing out such deals can sometimes lead to a crisper answers, while also removing one-year “make good” deals that could be signed with other benefits besides just money in mind.
It’s not that I was cherry-picking data. The OPP Premium still obviously existed in aggregate when looking at all players collectively, but it was best to ignore those one-year deals (or just small deals in general) in order to really identify the existence the OPP Premium effect. In fact, the decline of the OPP Premium that I’m reporting in this article would disappear if we didn’t limit our sample to the relevant subset of free-agent deals.
Another important aspect of the OPP Premium that I identified as the idea was being scrutinized by the sabermetric community is that “hometown discounts” given to players who sign contracts well in advance of free agency (in exchange for financial security) are a huge part of the OPP Premium. Since their cases don’t feature the same cases of information asymmetry on which I’ve focused, I think that excluding deals signed more than a year before free agency is the best approach, as well. This lowers the OPP Premium of course, but it better reflects the truth.
When we compare the OPP Premium in the “pre-THT Annual” and “post-THT Annual” time periods (loosely, 2006-11 versus 2012-16), we see that the more precise the subset of contracts reviewed, the more pronounced the decline in the premium. The last column shows that on the most relevant subset of contracts — multi-year deals over $2 million above the league minimum salary, signed less than one year before a player reaches free agency — the OPP Premium declined from 20% to 7%. Besides the peskiest deals (those under $2 million, for which I frequently have poor information on incentives), the OPP Premium declines for every other subset included in the sample.
| Year Range | All Players | Net AAV >= $2MM | Net AAV >= $2MM, Multi-Year Deals |
Net AAV >= $2MM, Multi-Years Deals, Signed w/in 1 year of FA |
|---|---|---|---|---|
| 2006-11 | 15% | 25% | 31% | 20% |
| 2012-16 | 16% | 21% | 20% | 7% |
The table below breaks down the information contained in the last column. Note that teams spent significantly more on their own players over time, and that was the driver closing this gap. This lends some important credibility to my claim in the first article in this series that the effect of teams learning more information leads them to bid up undervalued players as much as it causes them to lower their spending on overvalued players.
Indeed, we see that OPP cost per WAR did actually go up from the pre-period to the post-period, albeit slower than the rapid increase for re-signed players.
| Year Range | Player-Seasons Re-signed | Player-Seasons OPP | $/WAR Resign | $/WAR OPP | OPP Premium |
|---|---|---|---|---|---|
| 2006-11 | 322 | 493 | $6.09 | $7.29 | 20% |
| 2012-16 | 277 | 409 | $8.40 | $9.00 | 7% |
When I drilled down into the details of this finding, I contrasted the OPP Premium for hitters and pitchers and found that it was quite different. Pitchers are frequently injured and frequently decline in skill level suddenly, so private information about a team’s own pitchers made teams uniquely able to eschew re-signing their riskiest pitchers. The OPP Premium was more than twice as large for pitchers as for hitters. When I wrote about this finding earlier, I frequently noted the importance of this distinction.
Fascinatingly, the OPP Premium for pitchers has completely disappeared and reversed itself, while the OPP Premium for hitters has increased. I suspect that this is just a function of limited sample sizes, but it could also be a case of the market overcorrecting for the risk of signing other team’s pitchers.
| Year Range | Player-Seasons Re-signed |
Player-Seasons OPP | $/WAR Resign | $/WAR OPP | OPP Premium |
|---|---|---|---|---|---|
| 2006-11 | 185 | 251 | $5.92 | $6.67 | 13% |
| 2012-16 | 160 | 211 | $7.25 | $9.51 | 31% |
| Year Range | Player-Seasons Re-signed |
Player-Seasons OPP | $/WAR Resign | $/WAR OPP | OPP Premium |
|---|---|---|---|---|---|
| 2006-11 | 137 | 242 | $6.39 | $8.22 | 29% |
| 2012-16 | 117 | 198 | $11.13 | $8.45 | -24% |
I think the best evidence is that teams have gotten smarter about signing free agents from other teams, and the market has become more efficient as a result. While it is difficult to say for sure due to sample sizes, the marked decline in OPP Premium for pitchers in particular suggests teams now consider the information asymmetry when targeting free agents.
While, as I say, the OPP Premium is probably the most significant outcome of my work on free-agent market inefficiencies, there are several other sources of variation in cost per WAR that I have found, as well. The subsequent articles in this series will analyze changes over time in these discrepancies.
Matt writes for FanGraphs and The Hardball Times, and models arbitration salaries for MLB Trade Rumors. Follow him on Twitter @Matt_Swa.
I’m not sure if this research was also done (or inspired) by you, but I’ve seen similar research that reached the same conclusions about NFL free agents as well.
So you’re saying you’re not down with OPP?
I’ll create my own damn song. Titled O.P.O.P.P. (Over Paying Other People’s Players)
I assume physicals are becoming more detailed, especially for pitchers. I want to say Astros have backed out of a couple of pitcher free agent deals, Aiken in draft, and Cionel Perez international deal (though reworked for less money). Aiken and Perez don’t fit OPP, but lesson of needing to be careful of signing pitchers without knowing a guy as closely as if he was on your team is similar.
Still think it’s tough to differentiate the OPP Premium effect from the Hometown Discount effect, even with eliminating the extensions signed more than one year from FA.
You’re right to be skeptical and many people were. It is part of what drove me to do the Hardball Times Annual article in 2012. In that article, I showed that:
(a) players signed from other teams also underperformed projections and re-signed players overperformed projections
(b) traded players underperformed projections relative to untraded players
(c) traded prospects underperformed their BA rankings relative to untraded prospects
The concept that players could take discounts this large even in their walk year seemed less likely to me, but since that was not based on any data, I think it was important to compare players to non-market determined baseline as well to corroborate the finding. That said, the remaining 7% gap could plausibly be attributable in part to hometown discounts now.
Thanks, very interesting.
“Fascinatingly, the OPP Premium for pitchers has completely disappeared and reversed itself, while the OPP Premium for hitters has increased. I suspect that this is just a function of limited sample sizes”
You hit the nail on the head re: sample size. Unfortunately, it pretty much invalidates your entire theory. If a measurement goes from +29% to -24% in two small samples, a logical person would conclude that there is no evidence that the measurement was ever non-zero. What makes you so sure you haven’t been observing variance the entire time?
Oh great point! I totally forgot the fundamental rule of statistics: any sample size is too small if it can be split up into multiple too-small sample sizes. It’s no wonder polls are wrong! It’s just a bunch of sample sizes of one!
Now I know that the fact that the full sample was almost four times larger, and the fact that the conclusion held looking at different samples of traded major leaguers, traded prospects, etc. is all invalid because less than a quarter of the original sample produces a quirky result.
Thanks for being there!
Unfortunately, sarcasm doesn’t make bad science any better.
I’ll ask again: What makes you so sure you haven’t been observing variance the entire time?
Your one out-of-sample test failed miserably to validate your hypothesis. In any other world, this would be acknowledged as a serious red flag. In yours, somehow, it’s merely “fascinating.”
There are plenty of out of sample tests that upheld the result and I told you about them. That splitting the post-period sample produces a quirky result does not change that.
The entire point of the article was that there may have been a change in how teams are thinking about these things. From actually talking to teams, I know that this is something that has been on many team employees minds. So it may yet be a market over-correction.
But the results held before the initial publication, and held using:
1) production per dollar for hitters,
2) production per dollar for pitchers,
3) projections for hitters,
4) projections for pitchers,
5) traded MLB hitters,
6) traded MLB pitchers,
7) traded minor league hitters,
8) traded minor league pitchers,
It also held for variations of the original production per dollar posts on two-year deals, three-year deals, four-year deals, and longer deals.
This result has convinced skeptics with far more coherent skepticism than yours.
Oh, now I remember. You’re the guy who created SIERA (a.k.a., a cautionary tale of over-fitting). When the signs of some of your coefficients flipped between two different samples, you still thought they were important enough to keep in your formula. [Sigh].
“There are plenty of out of sample tests that upheld the result and I told you about them”
You concluded from your first research that the premium was 20%. An out of sample check showed 7%.
You concluded from your first research that pitchers received a much higher premium. An out of sample check showed the exact opposite.
How do you know that 400 player-seasons is a sufficient sample from which to draw firm conclusions? I’m not seeing any discussion of standard deviations and error bars.
I think this is a productive conversation, and I’d like to hear the author continue to defend his theory against reasoned critique. What I don’t get though is why people are downvoting the author’s responses. Sure, he’s being a little snarky, but this is the internet for chrissakes. Let the man talk, and if you think he’s wrong, then follow evo’s example and write why.
Once the prior commenter started attacking other unrelated articles, I stopped replying. You seem skeptical but interested, which is my favorite type of commenter and I’m happy to engage. One thing no one has ever criticized me about before is ignoring criticisms in comment sections.
The only thing that would have changed my conclusions from the previous work is more findings from before the analysis became published. Once the article was published, it became pretty well known and many market changes were occurring, even outside of this concern. This article was to see if the market changed, so evidence from the post-period would not affect my beliefs about the pre-period. I went from front office employees telling me they were shocked about the conclusion several years ago, to front office employees asking me about the conclusion more recently.
The pre-period evidence was overwhelming, and I listed much of it above. The $/WAR gap held for SP, RP, and hitters. It for 2-year, 3-year, 4-year, and longer details. It held for individual years in those multi-year deals. While determining standard deviations for $/WAR is tricky, the actual hitter and pitcher performance relative to projections was easier to analyze and those were statistically significant. The findings held when looking not just at $/WAR, but at projections for those players– both playing time projections and rate stat projections.
Plus the same finding held for several independent and unrelated samples. It held looking at projections for traded MLB hitters, traded MLB pitchers, traded hitter prospects, and traded pitcher prospects.
The hypothesis held up to a very large number of groups of players.
Furthermore, other researchers tested this and came to the same conclusion as well, including some of the biggest skeptics of the original post. A commenter above noted that he had read the hypothesis held true in the NFL as well. If that’s true, we have yet more evidence.
If anyone is skeptical of the original finding, I encourage you to get the 2012 THT Annual. Not only will you likely be persuaded by that piece, but you will get amazing work by other authors that will justify the purchase if not. Everything was good in there, but one in particular the stands out is Brian Cartwright’s analysis of HitFX data, which I think was amazing seminal work that has aged very well.
https://www.amazon.com/Hardball-Times-Baseball-Annual-2012/dp/0879464747
I’ll respond to any questions about that piece as well, privately or publicly.
In the post-period, it is clear that the original finding may no longer hold. Teams may have wised up. That is what I was testing in this piece. It doesn’t change my opinion about the period of time before the piece was published, because of the overwhelming evidence that such behavior existed at the time. If it were only a few hundred players and subsets of that groups did not all point in the same direction, I would have to revisit that conclusion. But that group had already been subsetted into many different conclusions. Many people questioned the initial research, but I have yet to have anyone express skepticism after reading the THT piece.