FG on Fox: The Over and Under Achievers
There are six weeks left in the 2014 regular season, and if the season ended today, we’d have some fairly surprising playoff teams. The Brewers are in first place in the NL Central, the Orioles have a huge lead in the AL East, and the Mariners are tied with the Tigers for the second Wild Card spot. You probably didn’t predict any of those outcomes, and I know I certainly didn’t. This is part of what is great about baseball, and especially in the current age of parity, the playoff teams are no longer as predetermined as they once seemed.
Results like these often convince people that preseason forecasts are basically useless. As you’ve probably been told repeatedly by various announcers and baseball scribes, the game is played on the field, not on a spreadsheet. However, I thought it would be instructive to look back at the forecasted performance from the beginning of the season and see how well they managed to evaluate expected performance.
To do this, however, we’re not going to compare the projected standings to the actual standings, because a team’s record is essentially a function of two things: how many hits, walks, and other positive events a team creates relative to how many they give up, and the timing of when those events occur. The first one is what projection systems specialize in forecasting, but they really have no way of knowing which teams will tend to bunch their hits together, or distribute their runs in such a way as to win a bunch of close contests.
The timing aspects of win-loss record is basically random, and since there’s no real way to project it in advance, we don’t really want to judge how well a projection did based on results that are influenced by randomness. Instead, we’re better off looking at just the quantity and value of the types of baserunners a team achieved over and above what they gave up, and evaluate the preseason forecasts based on how well they match up with what a team’s expected record would be without the timing effects that can skew runs and wins. After all, that is really what the forecasts are trying to measure.
At FanGraphs, we publish the seasonal data from a model called BaseRuns, which takes all of the events a team creates and allows and turns them into an expected runs scored and runs allowed total. Based on those numbers, we can come up with an expected winning percentage that doesn’t factor the timing of events into the results, and so that’s what we’ll use to measure the team forecasts.
Here are the top five teams that have outperformed their preseason expected winning percentages.
Dave is the Managing Editor of FanGraphs.
Rangers BaseRuns projected #6org
Sorry.
That puts the Rangers in the bottom 0.0536 percentile of awfulness.
Good thing the Royals don’t read this site or they might have traded Shields instead they are in first place.
Of course, if they had read this site, they wouldn’t have gotten Sheilds in the first place and would be enjoying similar production from Odorizzi for years to come and they would have Will Myers and a lot more money.
The Red Sox had bad injury luck this year?
Just because we don’t yet understand the sequencing of events does not make them entirely random as this site frequently asserts. This is an area for research with the potential to actually improve forecasting.
Throwing your hands up, declaring randomness for all the unexplained variation, and then evaluating forecasts against batting outcomes rather than the actual win-loss records is a bit perverse. You begin the article by stating that people commonly think the forecasts have little value. Your argument is essentially: “Well if we ignore all the parts of baseball that we don’t understand and just concentrate on the parts of baseball that we have a bit of a handle on, then the forecasts do OK”. ….well, yeah, but that wasn’t the question, was it?
I think everyone here understands the value of the projection system. It is accurate most of the time, but it is arrogance to assume that all variables are accounted for in the projections and to say that all variance is due to luck. The Orioles have out performed their projections drastically two of the last three years. Is there something in their construction that could lead to repeated success? There are many avenues for investigation here and I think they would lead to interesting articles. Far more interesting than, “It’s all luck and you’ll regress”. I’m reminded of Avery Pennarun’s post on the Curse of Smart People .
(…except for that one year when the Orioles didn’t.)
I flipped that coin and it came up heads 3 times in 4 tosses! It can’t be a fair coin, can it?
Next year, you can have the O’s outperforming their BsR win projections by 5 games or more, I’ll take the under, for any amount. Deal?
Yes, but in the case of a coin, all variables are under your control. This is not the case of the Orioles. Most of the time these uncontrolled variables have an insignificant effect on the outcome or they may cancel each other, but when they don’t we have an opportunity to study them. I’m not ruling out luck as the cause, but it would be ignorant to do so without further investigation. Clearly, there is a reason they are out performing. My guess is it is related to a dominant bullpen coupled with power hitters. The bullpen aspect of this is difficult to control.
For the record, in the third year, the Orioles did outperform their projections, but not to the degree they did in 2012 and have in 2014.
If I had assurance that the team make up was the same and the bullpen would again be dominant, I’d take the over. Of course, unless the Orioles kidnap Craig Kimbrel and clone him 5 times, there is no way to guarantee this.