What Do Power Rankings Tell Us?
The idea of ranking every team’s power in a context-neutral situation is something that appeals to many of us sports fans, regardless of how we feel about statistics. The idea of listing each team in a descending order, such that if #4 played teams #5 through #30, we would expect #4 to win, has a certain appeal to most people – it greatly simplifies the MLB, complicated by the two separate leagues and the three divisions in each league.
There could be other measures of power besides future play, such as the strength of a team’s play so far, which is what Beyond the Box Score’s Team Performance Index attempts to measure. Sometimes we get numbers that don’t quite match up with the win-loss records that we’ve seen on the field. Yes, the Astros are still bad, but TPI has the Athletics, a sub-.500 team, at #11, and the Angels, a team that was seven games better at the time of the rankings, at #19.
Unless it’s the first week of the season, however, the “subjective” power rankings you see at megasites like ESPN, Fox Sports, or FanHouse are unlikely to do much other than rank the teams by win-loss record and then adjust that for their records over the previous week of games. Take a look at this graph, generated from the June 21st or 22nd rankings from each of the sites listed:
Click to enlarge
The black line represents a ranking of the teams by victories as of June 21st, with victories in the last 10 games as the tiebreaker. The black boxes represent FanHouse rankings, blue represents CBS Sports, and red represents ESPN. Unsurprisingly, there is little difference from the ranking by simple wins and the ranking from the experts and these sites. What differences we do see can likely be accounted for with the biases of the writer(s), whether or not the teams are on cold or hot streaks, and their position within their division.
The idea of the power ranking should be able to set us up with some interesting discussions. At the most prominent places in which they’re seen, however, the power rankings are nothing more but a slightly modified league standings page, with a comment on what happened to the team over the previous week. The ranking really has little to do with how these teams would fare against each other over the rest of the season.
We should be able to do better. My ideal power rankings would simply rank the projected strength of each team for the rest of the season – in that sense, power is actually ranked, unlike simply using wins and volatile streaks to put teams in order. There could certainly be many different methodologies, just as there are a variety of projection systems for individual teams. As long as we can get to something deeper than simply pointing out which team has more wins than the other teams, I’ll be happy.
Jack Moore's work can be seen at VICE Sports and anywhere else you're willing to pay him to write. Buy his e-book.
The problem with that approach is that projections vary so much from system to system. I mean, I can pull numbers out of a hat and call that a projection. It would be a bad projection, but a projection nonetheless. So it’s hard to validate the credibility of such a methodology, especially when most projection systems are black boxes. Using past performance as a future indicator, meanwhile, at least grounds the numbers in reality.
Not all projection systems are black boxes.
You can use previous years’ predictions vs. actual results to evaluate the systems. Most sites already do this. So yes, you can pull numbers out of a hat “and call it a projection”, but it’s not a particularly robust prediction, and upon future retrospective analyses it would be shown to be worthless.
Prediction systems DO use past performance as a future indicator. They just do it in a more sophisticated manner.
PS – is there any feature available where I can get an email if someone replies to one of my comments? I find it hard to remember to return to continue the conversation.