Revisiting the Champs and the Projections
Yesterday, I ran an exercise on this site that required some audience participation. The premise was simple enough: the Royals, for the third consecutive year, haven’t looked like an elite team based on the third-party projection systems we host here on the site. The Royals, of course, have been an elite team, despite what the projections say, so there’s been some understandable hesitation in taking those projections at face value.
I simply asked everyone to take a look at each individual player projection, and either take the over, the under, or push. The idea is that, through crowdsourcing, we might be able to spot the individual places where the community thinks the projections are missing on guys, and then manually adjust the team projection from there.
And now for the seemingly ever-necessary reminder: The projections are not meant to be taken as gospel. They’re to be used as a guide. Anyone who reasonably understands what we do here on FanGraphs should get that, by now. We — we being the authors of FanGraphs — have no say in the projections. Just because the numbers say one thing doesn’t mean that every author has to agree. I can’t change the fact that the projections say what they do. I’m just here to report, and analyze, and think, and discuss.
The numbers are calling the current Royals roster a 78-win roster. That seems sort of silly. I think you’d be hard-pressed to find too many folks who’d think that sounds right. I’d certainly take the over, at least. Let’s see what the crowdsourcing results say. You should be able to click this image to view a larger version:
Each poll had somewhere around 800 votes, and for the most part, people think the projections look about right, on an individual basis. The biggest discrepancy between the projections and public perception is Lorenzo Cain, where the projections say he’s closer to a 3.5 WAR player, and the crowd thinks he’s probably closer to a 5 WAR player, like he’s been the last couple years. I side with the crowd there. The other big discrepancy is Wade Davis, where folks think the projection is at least a win shy. I’ll take the crowd’s side there, too — I tend to agree with the growing belief that public WAR models might be missing out on some of the value provided by dominant, high-leverage relievers.
Besides Cain and Davis, the differences between the projection and the perception were small — less than a win in each case. You guys might take the slight over on players like Eric Hosmer, Jarrod Dyson, Paulo Orlando or Kendrys Morales, but you’d also take the slight under on the projections of Salvador Perez, Yordano Ventura and Kris Medlen. This is all about what I was expecting. Add everything up, and the crowd takes the over on the projections, by about four wins. I’d wager that you do this exercise for any team and the crowd would end up taking the over by at least a little bit, because projections err on the side of caution and people have grand ideas in January. But either way, the results here have the crowd taking the over by four wins on the Royals projections, and that figure seems reasonable.
So that puts the Royals at 82 wins. A .500 team. Starting with the projections, and then adding in the opinions of a room of 800 baseball fans, including plenty of Kansas Citians, we’ve arrived at the Royals looking like a .500 team, on paper. My gut tells me that Royals fans would still scoff at that sentiment, and maybe even take offense, for whatever the reason is that people choose to spend their energy getting angry at computers.
The most common complaint you hear from the anti-stat crowd is that stats are missing out on the value of clubhouse chemistry. And while I don’t think that’s entirely true — better chemistry would lead to better numbers, and the numbers are what fuel the projections — I totally see the argument, and I think it’s got merit. I’m a firm believer in the value of chemistry, though I prefer to think of it as confidence, though I suppose could be bred from chemistry. Whatever. Semantics. Confidence is vitally important to success in any walk of life, and sports are no exception. Anyone who watched the Royals last year knows damn well they aren’t lacking in confidence. On the contrary, I’m not sure how much any team could be — it’s tough to make it as a professional athlete without confidence — but with the Royals, there’s no questioning the chemistry or confidence. It was palpable. I’m willing to give them a free win or two not captured in the projections for chemistry or confidence or whatever you want to call it. I’m probably willing to give them another win or two due to whatever value we may be missing in the value of having an elite bullpen, but part of that was already captured in the crowd taking the over on Wade Davis’ projection.
So now we’re at four wins over the projections, based on the crowd taking the over on Cain and Davis and a couple other guys, and maybe another three wins or so based on the human element or chemistry or bullpen leverage or whatever you think it is that makes the Royals projection-proof. That’s seven wins over our initial 78-win projection, and an 85-win team is probably starting to get near the area that would please Royals fans.
But here’s the other thing. I just want to publish an image made by Jeff Sullivan around this time last year. He looked at teams that had beaten their preseason projections by more than 10 wins, as the Royals did last year, and then looked at what happened to those teams the following year. The result looked like this:
There just isn’t any historical precedent of teams being able to routinely beat their projections. That the Royals have done it in consecutive years may seem convincing in the present, but two years of a single team versus all of recorded history is a relatively small data point. And Jeff’s post reminded me that we just had these same conversations about the Orioles for a few years, with folks convinced they were just special, until they stopped being special. Same with the late 2000’s Twins, and the mid-2000’s Angels, and plenty more.
I can’t put my next thought into words any better than Jeff already did, so I’ll just quote him here, with one slight change:
This isn’t me being stubborn; this is me just not finding enough precedent. It’s difficult to believe the
OriolesRoyals have just figured something amazing out. It’s not impossible, but they might well be the first.
I want to make it clear that I don’t hate the Royals. I don’t hate any team — that would be a real weird thing for a full-time baseball writer to do. Quite the opposite, in fact; I’ve probably enjoyed my time watching the Royals these last couple years more than any other team. And to be quite honest, I hope they keep it up and smash the projections again. It’s healthy to find reasons to challenge common beliefs. I honestly don’t think there’s a right or wrong side here — you shouldn’t trust anyone who claims predicting the future to be an exact science.
You were all willing to give the Royals an additional four wins based on some potentially low projections, and I’m willing to give them a couple more on intangibles. It’s easy enough to find justifiable reasons for the Royals to be beat their projections, but then again, they aren’t the only team with confidence and a good bullpen and some bearish projections here and there. As reasonable as it is to find justifications for why the Royals might again beat their projections, one might argue it’s just as reasonable to look at the historical precedent that says they won’t. None of this would be any fun if they didn’t actually play the games.
August used to cover the Indians for MLB and ohio.com, but now he's here and thinks writing these in the third person is weird. So you can reach me on Twitter @AugustFG_ or e-mail at august.fagerstrom@fangraphs.com.
I’m sure someone somewhere has argued that the Royals have just figured something out (via clubhouse chemistry/undervalued talent, etc.) but the arguments I’ve seen are more along the lines of the projections just do a poor job of evaluating the Royals.
In other words, the Royals have been one of the ten best teams in baseball each of the last three years by most measures (WAR, for one example), and the projections see a sub .500 team that continues to outperform its skill level in the regular season in ways that cannot be expected to continue.
One thing to add is that the projection systems haven’t necessarily missed low on every Royals player over the last three years (they missed high on Infante/Rios/Guthrie/Duffy in 2015) but they’ve missed on pretty much everybody. I attribute this to a non-traditional developmental path taken by a bunch of players on the Royals roster that makes them difficult to project for systems that still remember replacement level seasons in the not distant past by key contributors.
Missing consistently on the Royals defense and bullpen is a bit more egregious and probably should have been corrected by this point. The team’s runs against projection is somewhere between .2 to .4 runs per game too pessimistic every year, despite being close to the mark on the value of the teams starting pitching. That’s a staggering margin to miss by year after year.
The Royals are ONE DATA POINT.
You can’t trash the system because it gets one bullpen wrong. If you want to make the case it gets all bullpens wrong, make that case. We’ll run the numbers for everyone and see where the Royals end up.
You can’t trash the system because it gets Royals non-traditional players wrong. If you want to make the case that it gets all non-traditional players wrong, make that case. We’ll run the numbers for everyone and see where the Royals end up.
But the problem in both of these instances is… Royals homers aren’t making that case. They’re just whining that the Royals aren’t declared projection champs. They want the output to say Royals are the best.
I am not saying the projections are perfect. Merely that you need to do better than saying “BUT THEY GOT THE ROYALS WRONG!!” if you want to show there is a flaw in the system.
Have the Orioles not outperformed their projections four straight years?
I realize they missed the playoffs last year so this comments section is populated by Royals partisans and not Baltimore fans, but the 2015 Orioles weren’t a win for projections, I don’t believe.
Futhermore, I thought the Orioles argument wasn’t so much a defense of projections, but rather that the 2012 Orioles did not discover the secret of winning one run games at a historic rate. As far as one run games go, the Royals win them at roughly their normal win percentage (where they really drive Pythagoras crazy is in two run contests), so this isn’t really like the Baltimore situation at all…except that the systems just don’t handle the defense and bullpen heavy teams very well.
To say the projections, “missed”, is to misunderstand how projections are really a range of outcome. The single line you’re comparing to is actually a range of projections with likelihoods.
Let’s say each of the last 2 years, these “projections” had the actual Royals outcomes as the 10% likelihood. As long as ~10% of the outcomes were around that outcome (for them), the “model” is still accurate.
It is possible the Royals are systemically mis-projected, but looking at the outcome of one team cannot prove it.
One reason to say that the Royals are going to be somewhat better than projections is that projections based on WAR are going to underestimate the importance of great relieving, which the Royals clearly have. Typical evidence that WAR as presently calculated isn’t accurate for relief pitching was a recent Fangraphs article about relief pitchers and the Hall of Fame in which it was simply assumed that the cutoff level in terms of WAR in terms of what is necessary to get into the Hall is much lower than it is for other players–no one even argued with that, so there must be general recognition that there is a significant problem here. Another reason in terms of offense is that the Royals have had the highest Clutch rating in the major leagues two years in a row (and they were #5 in 2013). While this may be a matter of luck, there is some evidence that high Clutch ratings are correlated with low K% (the Royals are extremely low in terms of K%), and teams with high Clutch ratings will naturally outperform their WAR-based projections. Aside from this, it also seems reasonable to suspect that there is a problem in how offensive WAR is calculated in that it doesn’t give any value to having a low K%–whereas people have generally accepted the idea that K% plays a significant role in calculating pitchers’ WAR, this is not done for a hitters’ K%, even though exactly the same arguments suggest that it should be. A third reason might be (although this more speculative) that the statistics used to calculate fielding WAR generally underestimate the importance of fielding, and because the Royals have outstanding defense, this is another reason that the Royals have significantly outperformed their projections in the last few years.
More specifically about bullpen: Projections based on WAR are going to underestimate the importance of great relieving, which the Royals clearly have. Typical evidence that WAR as presently calculated isn’t accurate for relief pitching was a recent Fangraphs article about relief pitchers and the Hall of Fame in which it was simply assumed that the cutoff level in terms of WAR in terms of what is necessary to get into the Hall is much lower than it is for other players–no one even argued with that, so there must be general recognition that there is a significant problem here. Another reason in terms of offense is that the Royals have had the highest Clutch rating in the major leagues two years in a row (and they were #5 in 2013). While this may be a matter of luck, there is some evidence that high Clutch ratings are correlated with low K% (the Royals are extremely low in terms of K%), and teams with high Clutch ratings will naturally outperform their WAR-based projections. Aside from this, it also seems reasonable to suspect that there is a problem in how offensive WAR is calculated in that it doesn’t give any value to having a low K%–whereas people have generally accepted the idea that K% plays a significant role in calculating pitchers’ WAR, this is not done for a hitters’ K%, even though exactly the same arguments suggest that it should be.
Oops, sorry, I didn’t mean to make similar posts twice, I thought the first one hadn’t been entered.
My only thing with that is that the Royals clearly aren’t done with the offseason. There is absolutely no way Dyson and Orlando are going to be starting every game in left and right field, at the very least. Reports have been saying that they are looking for both outfield and starting pitching. I’m not saying that free agents X and Y will be the difference between .500 and the ’01 Mariners, but this isn’t going to be the team that is fielded on opening day.
It’s a very good exercise, but I think it’s happening too soon when we could be giving better insight on what kind of team they’re going to be in March when the roster is more or less set.
I’m sure that the Royals contact ways, combined with such a strong back end of the bullpen, spurred SOME of their ability to beat there projections and baseruns. However, I’m guessing that those effects are very slight and most of it is just noise.
As August said we literally just had the same thing happen with the Orioles. It’s funny actually right up until the Orioles got eliminated in the playoffs last year everyone was up in arms about the Orioles “figuring something out”, and right when they got eliminated all Oriole narratives shifted to the Royals where they still exist. We saw what happened to the old poster boy Orioles last year.
So while it’s possible the Royals are built to beat their projections/baseruns slightly, I think the vast majority of it is noise. I wouldn’t be ALL that surprised to see the Royals win 76 games under-preforming the projections, and then have the Twins win 88 and have some success in the postseason;while beating their projections for the second straight year and becoming the new poster boy.
There were a couple of years (06-09ish) where the Angels and Mike Scioscia had it all figured out and the Angels beat their run diff every year. They had the secret to beat the projections and they were special, different, etc., until the Orioles apparently stole the short-lived magic.
It isn’t the same thing. The Orioles have been average in terms of K% and slightly above average in defense in recent years, while the Royals have been the best in the major leagues in both categories.
So, go do an analysis of all teams in history that were above-average in terms of K% and defense and see how they did in relation to expectations
I just don’t get this line of thinking. The Royals beat the projections, so there must be a reason why. What are the Royals good at? TO THE LEADERBOARDS! They are good at K% and defense. That must be why they beat projections!
To me it’s always been pretty apparent why they’ve been this way the last 2 years… players that, on the surface, don’t seem very good end up playing very well. When your players play well, you win games. When your players weren’t expected to play as well, you beat projections.
In terms of K%, as I mentioned before, there have been articles on Fangraphs recently showing that Clutch is significantly affected by K% (lower K%, higher Clutch), so this is a clear example of outperforming expectations, in addition to which (also as I mentioned before), if we’re going to accept the idea that K% affects pitcher WAR–which we do–we should also accept the idea that K% affects higher WAR–which so far we don’t. This just seems self-evident. Defense is not proven–this is something which should be looked at, especially considering that weighting defense WAR really is an open question.
Another thing differentiating between the Royals and the Orioles is that in terms of relief pitching, even though they have appeared to be relatively close in terms of WAR, it’s misleading because the Royals’ bullpen strength has been much more concentrated in terms of three great pitchers while the Orioles has been more the bullpen in general. For example, in 2014, the Royals total bullpen WAR was 5.0 (4th best) while the Orioles was 4.5 (7th best). But when it came to individuals, for the Royals Davis was 3.0, Holland was 2.2, and Herrera was 1.2, while the top three Orioles were Britton (1.0), Miller (0.9), and O’Day (0.9). The Royals’ top three were #2, #9, and #25 in the major leagues in relief pitcher WAR–the top Oriole was Britton at #33. This means that the Royals could use very strong pitchers in high-leverage situations whereas the Orioles couldn’t, and this again means that the total relievers’ WAR underestimate the actual value of the Royals’ bullpen relative to the Orioles’. There was also an article in Fangraphs recently about the effect of elite relievers on performance relative to WAR.
The general point here is that the Orioles have been a team which does not have any of the features which are likely to be associated with WAR underpredicting results, so it is not surprising that they have regressed to the mean. On the other hand, the Royals do have these features, so they are in a different situation.
So what you are saying is that Orioles magic was just an illusion, while Royals magic is Real Magic!
Unfortunately, even FGs has fed this line of thinking a little too much. Jeff ultimately found almost no connection between K rate and clutch (the r squared was like .02) but it doesn’t stop some people from trumpeting this as some kind of material factor. Same thing with the elite bullpen theory. The evidence is extremely thin, but the claim is now repeated as fact.
So much this. It really gets annoying.
It seems that this is about the 100th article in the last two years talking about how Royals keep beating projections. It seems that after a certain number of observations of this some work would be done to adjust the team-based win projection systems.
E.g. How about adjusting team based win projections based on a few things that everyone generally agrees upon:
I. DEFENSE
1) A very strong defensive team turns poor pitchers into average pitchers and average pitchers into solid pitchers (at least according to old school results-based stats). This means less pitches per inning, less pitches per game and the ability to go slightly deeper into games then we would expect, and likely less fatigue over the year. Over the course of the year this must lead to about 3 Wins extra. This likely leads to pitchers showing lower BABIP numbers and thus less WAR but in regards to the number in the win column it must help a lot.
II) BULLPEN: It’s already been shown that relievers WAR projections are lower than what should be the case, with their innings not being as valued as starters. The Royals have shown over and over that having an elite bullpen allows them to get the most of late-inning, high-leverage situations. The projections don’t care about whether you are more likely to win close games due to an elite bullpen but the Win column does. Keep in mind that the Royals are not the first to figure this out. The 1990 Reds won a championship based on the 3-headed monster of Myers, Dibble and Charlton. In close games you don’t know which of the last three innings will be the toughest and have a third lights out reliever (instead of the old focus on just a single “closer”) means you can outperform your opponents in close games. This bullpen “chaining” must lead to at least 4 wins over what their individual stats show.
III) CONTACT CHAINING:
a) We know the effect of having like minded hitter profiles leads to better results than building a team of different types of hitters. The Royals are built around contact and thus there should be some adjustments to a team’s projections when they are built around a single core strength. You can add another 2 or so wins here.
b) There is also a number of studies showing that contact rate neutralizes the impact of facing high-strikeout pitchers somewhat (which many elite closers are). The reason contact hitters tend to do better in high-leverage situations is likely due to this. Again you can add a couple wins in close games due to this.
IV) SPEED: A team with great speed can take advantage of a large ballpark by turning doubles into triples and triples into Inside-the-Parkers. When combined with contact hitting it must multiply the impact of both skills as:
a) holding fast runners on leads to larger gaps for contact hitters, and thus a higher BABIP
b) the likelihood of rally killing double plays decrease due to the extra speed
c) the likelihood that runners advance an extra base is greatly increased when contact and speed are combined.
d) High leverage situation: needing that one less hit due to having a pinch runner like Dyson who can turn a runner on first situation into a runner being on second from a SB and then scoring on a single. In tied games or very close games this has been a game changer.
It seems the projection systems miss out on the combination of speed and contact and their relationship with one another and their impact in close games.
IV) STADIUM IMPACT: We know the Royals play in the largest park in the majors that naturally gives an advantage to defensively strong teams. The Royals also happen to be a defensive-minded team with much outfield speed (the 1980’s Cardinals showed us how much of an impact this can have). This likely gives them 1-2 extra wins in their home games during the course of a year. Teams that build around a unique ballpark feature (e.g. the Red Sox having Right handed, flyball hitters who pull, or the Dodgers building around starting pitching) should get a couple wins added to their column.
When you take all these elements together you have a team that has seemed to build itself on maximizing every possible advantage you can have in close games thus leading to a team that wins more games than the numbers thinks they should. I think we simply need to start incorporating these known factors into the projection system so that we can have projections that hold up better to the reality on the field.
1. Just because all the teams that beat their projections one year cannot the next year does not mean that SOME teams cannot do it consistently. Jeff’s random scatter plot has little bearing on whether the Royals specifically can consistently beat their projections.
2. Just because teams cannot beat their projections forever, does not mean that they never had some skill that allowed them to do it in the past. To me this is one of the strangest arguments. Everyone accepts that individual players lose their skills when their outcomes change. Why should we then expect a team like the Angels to be able to go on beating their projections forever if it is a skill? Clearly we shouldn’t.
Mind you, I am not arguing that some such skill exists. Rather, just that the arguments against it are seriously flawed.
High-leverage relief pitching is underrated. Tom Tango has suggested multiplying LI by the underlying performance to get an equivalent measure to SP, who typically have an LI approximating 1. The top four 2015 Royals RP provided 6.8 LI WAR (8.24 WPA). This accounts for part of the difference last year.
That being said, the 2016 Royals should project to be worse.The 2016 Royals bullpen is not as strong as last year. Ryan Madson and Greg Holland are gone. They gave the Royals 2.8 leveraged wins last year (2.9 WPA). Those two had a SD/MD ratio of 51/11. Hochevar and Morales, the next two RP, were 22/19. That’s a pretty big loss.
Alex Gordon is gone. He’s projected at 3.5 WAR for 2016. He’s replaced by Jarrod Dyson in Depth Charts. That’s a 2+ WAR swing.
There is a double whammy with the player losses because the Royals lose depth. Dyson goes from valuable 4th OF to below average starter. Same for the bullpen shifts.
And of course, there’s clutch and random variability. Say what you want but the Royals probably can’t keep it up. Harder to be clutch when you’ve shortened a bullpen that kept you in games.
But the Royals were 25-5 without Gordon, so they’re actually better without him. Stats!
I’m curious about your thoughts on the particulars of those high-leverage innings being underrated. First, at a general level, doesn’t RP fWAR take leverage into account? It could still not be ‘enough’, but what kind of magnitude are you suggesting? 5%? 10%? more? In order for 2015 Wade Davis, for example, to merely be one of the top 50 SP, you’d have to increase his WAR by about 30%. To make him top 25, you’d have to increase it by about 75%.
Second, if you are suggesting a significant boost to RP WAR, why is pressure that valuable in terms of player performance? Are you suggesting that performing in high leverage situations is a repeatable skill? Should Clayton Kershaw start pitching in the 3rd inning rather than the 1st inning because a lesser pitcher won’t be as intimidated by the lower leverage innings early on?
Third, why don’t teams pay RP that way? Mike Leake was about the 80th best SP by WAR last year, and he’s getting about $80 million guaranteed (plus another option year after that). Andrew Miller was a top 10 RP, and he signed last year for $36 million.
A small adjustment upward for reliever WAR makes sense to me, but I just don’t see how a major underrating is consistent with how ball clubs both use and pay starting pitchers.
Forgot. Soria should add back in the bullpen.
In any case, I would see the over/under at about 85 wins.
i don’t specifically hate the royals, they just seem to always face off against and beat the teams i’m rooting for in the playoffs and that has grown a sort of bitterness and resentment towards them inside me, though i do admit that they have impressed me and they are otherwise a fun team to watch.
I flipped 100 coins and 50 landed on heads. Of those 50, 25 landed on heads after another flip. Of those 25, 12 landed on heads yet again. Of those 12, 6 landed on heads once more. Of those 6, 3 landed on… Heads. Of those 3, 2 landed on heads after another flip. Of those 2, 1 landed on heads after one last flip.
That 1 coin landed on heads 7 times in a row.
Royals fans cite this as proof that said coin is weighted, that said coin is on to something, that said coin is special; after all, how could a regular coin land on heads 7 times in a row by chance? Sure it’s POSSIBLE, but the odds are too low for the presumed coin projection to be considered accurate.
One other thing to factor in is the AL central has been pretty weak the last two years, especially last year. They were 44-32 vs AL Central last year.
They sure look like a .500 team to me right now. But if they re-sign Gordon and add Kazmir, that would surely make them th Central favourites.
Continuing to enjoy this series. I’m feeling a bit contrarian on the relief pitching. I think people are over inflating the value of relievers due to the scoreboard results rather than the underlying value of the pitching. The Royals’ great strength has not been the RP. It has been the team defense. The pen has been valuable, but it’s the defense that has made them exceptional.
In addition to that, the Royals have also implemented a remarkably disciplined regimen. It literally got its own acronym: HDH. Rather than bringing in relievers in high leverage situations, Yost would bring in relievers to start an inning. I have no idea how to evaluate that impact, but it seems a variable to consider.
A third factor is the business side. Teams value pitchers that can pitch 200 innings a lot more than they value pitchers that can pitch 70 innings. That doesn’t mean front offices are ‘right’, but the widespread nature of the salary discrepancy is certainly suggestive. Moreover, the Royals in particular provide an almost perfect case study in directly comparing the value of a SP and a RP. Wade Davis was acquired to be a starter. And he failed. There’s no shame in that; he’s become one of the most interesting pitchers in baseball in the bullpen. But he is clearly less valuable – much less valuable – than if he were good enough to be a dominant starting pitcher.
Projections are not perfect, but finding random error and assuming it must be systematic error is a common mistake. The Royals were better than they were projected to be. Random error. Unless shown otherwise with large amounts of data, I’m going to assume it has nothing to do with bullpens, or K’s or defense or anything like that being undervalued by projections because all are included in projections. It should be presumed to be simply random error. I’ve seen the same thing with the Red Sox too. People assuming they will underperform their projections again because they have the lasat 3 years.
When you see the Royals projected as a .500 team, or the Red Sox projected as the best in their division, you can say projections aren’t perfect, which they aren’t. There’s a lot of random error. That’s the way baseball is. You can only get so close. But these are carefully crafted systems most likely free of systematic error. But the fact that those teams over/under performed their projections in past years has nothing to do with how they will perform relative to their projections in the future.