Translating Farm System Rankings into Wins

Knowledgeable baseball fans clamor for rankings of farm systems every offseason. Experts’ opinions are highly coveted, as fans eagerly await information on where their team’s prospects rank. This year, Baseball America ranked the Rangers atop its list, with the Royals just behind them. On the other hand, The White Sox’ and Indians’ farm systems were ranked at the bottom.

This year’s rankings saw big spenders like the Phillies, Dodgers and Tigers in the bottom-third in baseball, while the top-third included some teams with the lightest payrolls — like Rays, A’s and Padres. Obviously fans want their teams to be big spenders on veterans — and big farmers of young talent — but which is more important? What does a good ranking mean in the win and loss columns, and how much can payroll explain that performance?

Team building is at the heart of baseball analysis. And analysts like those here at FanGraphs try to deduce what teams should do to get themselves into the playoffs. Recently, I wrote about a couple of important team-building concepts at The Hardball Times. Specifically, I separated all WAR into either “Non-Market WAR” and “Auction-Market WAR.”

Non-Market WAR (NM WAR) refers to all players who weren’t yet eligible for free agency — so they were either making league-minimum salaries or arbitration salaries. Auction-Market WAR (AM WAR) refers primarily to players with at least six years of service time. The only exception is professional free agents, typically from countries in Asia or Cuba, since teams aggressively bid on them like free agents. The difference is that teams had to consider other teams as competition when paying for AM WAR — but NM WAR is homegrown.

Each year, free-agent-eligible players get about 75% of payroll, but they only produce about 30% of all WAR. The average team gets about 12 WAR from Auction-Market talent, but 26 WAR from Non-Market Talent. At the current price of free-agent talent, it’s effectively impossible to build a team out of auction-market talent alone. We already know that a team of replacement-level players would only win 43 games. The 2009 Yankees would be the only team in the past five years to have enough AM WAR (41.0) to be above .500 without any contribution from NM WAR—but they would fall short of the playoffs at 84-78. Put simply: You absolutely need some cheap talent to win.

On the other hand, there have been 13 teams in the past five years with more than 38 NM WAR — which is enough to break .500 without dabbling in free agency. The best of these teams was the 2011 Rangers, who got 47.8 WAR from players not yet eligible for free agency. Adding this to the 43 wins that a replacement level team would put together, that means that the 2011 Rangers could have reached 91 wins without spending a dime on free agents. This is a strong contrast with the 2007 White Sox, which only had 5.3 NM WAR. That means that even with the 2009 Yankees’ free-agent production, the 2007 White Sox still wouldn’t have won as many games as the 2011 Rangers’ amateurs would have won if you replaced all players with six years of service time with AAAA talent. So you can certainly make good use of free agents’ AM WAR, but it’s possible to win big with NM WAR alone. You just need to be both very lucky and very good at scouting and development.

Building from within is essential to team success, and that’s why farm-system rankings are so important to knowledgeable baseball fans. Do they tell us that the Rangers are the premier team to compete with over the next few years? Are the Royals the next juggernaut in the AL Central? To answer this, the next step is to quantify these rankings. To figure this out, I gathered the farm-system rankings from Baseball America for the past 10 years.

Sure enough, there is a sizable and obvious correlation between Baseball America farm system rankings and NM WAR in subsequent years. Baseball America ranked the Rangers fourth, first and second between 2008 and 2010, and the team subsequently led the league in NM WAR in 2011. The biggest producers who factored into the 2011 rankings were Elvis Andrus and Matt Harrison, but their NM WAR leaders were Ian Kinsler, Mike Napoli and C.J. Wilson.

The problem is that much of the variation in NM WAR is still unexplained—at best, about 7% of differences between teams’ NM WAR in a given year can be explained by differences in prospect rankings. The rest is either something that prospect gurus were missing or something that could not have been foreseen. Either way, rankings can explain a limited amount of this very important difference between teams. The Diamondbacks, for example, had more WAR from players who did not reach arbitration eligibility than any other team in 2011 (23.1 WAR). At the same time, their farm systems were ranked third, 15^th, 26^th, 27^th and 22^nd between 2007 and 2011. The Twins were ranked a respectable eighth, 18^th, 22^nd, 7^th and 13^th in the past five years, but they only had 11.5 NM WAR in 2011 —which was lower than any other team last year.

If you regress NM WAR in each of the next five years against a team’s Baseball America farm system ranking, you’ll find that the difference in NM WAR between the best and worst teams in the league is about 41 expected wins over five years. In other words, the difference between the Rangers and White Sox between 2012 and 2016 should be about 41 wins — plus or minus differences that can be expected from major league contributors and Auction-Market WAR. The biggest effect is two years later, where the gap is 10 wins (31 vs. 21 NM WAR). Of course, that’s the expected NM WAR based on rankings. Actual differences are much larger with all factors considered — the difference between the Rays and the Mariners, the best and worst teams by NM WAR in the past five years, was 122.1 WAR (194.2 and 72.1). It’s just that farm-system rankings weren’t going to tell you what the Rays would do that the Mariners would not.

So we have some numbers: Each team can expect about 1.4 more wins in the next five years than the team ranked below it, with the largest gap occurring two years later with about 0.33 wins difference for each spot in the rankings. That’s definitely important, but it’s hardly a crystal ball.

Contrary to past farm system rankings’ limited ability to predict future NM WAR, past payroll predicts future AM WAR very well. If you take payroll from 2007 and try to predict AM WAR for 2007 through 2011 through regression, the differences between the highest and lowest (Yankees and Rays) expected AM WAR goes down from 41 WAR in 2007 to 25 WAR in 2011. Overall, the difference is about 160 WAR in those five years. Even excluding the Yankees, the difference between the Rays and the second-highest team (Red Sox) is about 115 AM WAR.

Adjusting for payroll growth over the last five years, every extra $1 million of payroll today suggests about 0.74 more AM WAR in the next five seasons, combined. Annually, this effect starts at 0.19 more AM WAR this year per $1 million, down to 0.12 more AM WAR in four years.

Bringing these two things together, we see that differences between the top and bottom teams in farm ranking will accumulate to about 41 WAR over 5 years, with the biggest different being about 10 wins two years after the ranking. The difference between the top and bottom payroll rankings will be about 160 WAR over five years, gradually decreasing from 40 WAR that same year to 25 WAR four years later.

So if I were to tell you that you had the fifth-highest payroll in 2011 and the fifth-worst farm system, chances are you would get five more AM WAR in two years than the average team and three fewer NM WAR in two years than the average team in two years  —which would put your team around 83-79. If you were first in farm and last in payroll, you might expect to finish with five more NM WAR than average and about 13 fewer AM WAR than average — which would relegate you to a 73-89 finish.

That means it’s better to be the Phillies right now (third in payroll and 27^th in farm, implying an expected difference of +11 AM WAR, -4 NM WAR, versus average) than the Royals (25^th in payroll and second in farm, suggesting -6 AM WAR, +4 NM WAR, versus average). Of course, you’d rather be the Red Sox than either (second in payroll and ninth in farm, +11 AM WAR, +2 NM WAR, versus average), and you definitely wouldn’t want to be the Indians (24^th in payroll and 29^th in farm, -7 AM WAR, -4 NM WAR, versus average).

Payroll in 2007 can tell you about four times as much as about the difference between teams over the next five years as farm-system rankings can. As bright as the Rangers’ future may look, the Yankees future is probably a bit brighter. Even still, 41 wins difference between the top and bottom teams is not small. If the White Sox were to ask the Rangers to swap  farm systems, a fair price would be about $235 million. That’s useful, but the foundation of a big payroll — a large coastal city full of deep pocketed fans, a regional sports network and a retro-classic stadium— is worth a lot more than that. And now we have some numbers that tell us about how much.