What Do Power Rankings Tell Us?
The idea of ranking every team’s power in a context-neutral situation is something that appeals to many of us sports fans, regardless of how we feel about statistics. The idea of listing each team in a descending order, such that if #4 played teams #5 through #30, we would expect #4 to win, has a certain appeal to most people – it greatly simplifies the MLB, complicated by the two separate leagues and the three divisions in each league.
There could be other measures of power besides future play, such as the strength of a team’s play so far, which is what Beyond the Box Score’s Team Performance Index attempts to measure. Sometimes we get numbers that don’t quite match up with the win-loss records that we’ve seen on the field. Yes, the Astros are still bad, but TPI has the Athletics, a sub-.500 team, at #11, and the Angels, a team that was seven games better at the time of the rankings, at #19.
Unless it’s the first week of the season, however, the “subjective” power rankings you see at megasites like ESPN, Fox Sports, or FanHouse are unlikely to do much other than rank the teams by win-loss record and then adjust that for their records over the previous week of games. Take a look at this graph, generated from the June 21st or 22nd rankings from each of the sites listed:
Click to enlarge
The black line represents a ranking of the teams by victories as of June 21st, with victories in the last 10 games as the tiebreaker. The black boxes represent FanHouse rankings, blue represents CBS Sports, and red represents ESPN. Unsurprisingly, there is little difference from the ranking by simple wins and the ranking from the experts and these sites. What differences we do see can likely be accounted for with the biases of the writer(s), whether or not the teams are on cold or hot streaks, and their position within their division.
The idea of the power ranking should be able to set us up with some interesting discussions. At the most prominent places in which they’re seen, however, the power rankings are nothing more but a slightly modified league standings page, with a comment on what happened to the team over the previous week. The ranking really has little to do with how these teams would fare against each other over the rest of the season.
We should be able to do better. My ideal power rankings would simply rank the projected strength of each team for the rest of the season – in that sense, power is actually ranked, unlike simply using wins and volatile streaks to put teams in order. There could certainly be many different methodologies, just as there are a variety of projection systems for individual teams. As long as we can get to something deeper than simply pointing out which team has more wins than the other teams, I’ll be happy.
Jack Moore's work can be seen at VICE Sports and anywhere else you're willing to pay him to write. Buy his e-book.
The problem with that approach is that projections vary so much from system to system. I mean, I can pull numbers out of a hat and call that a projection. It would be a bad projection, but a projection nonetheless. So it’s hard to validate the credibility of such a methodology, especially when most projection systems are black boxes. Using past performance as a future indicator, meanwhile, at least grounds the numbers in reality.
Not all projection systems are black boxes.
You can use previous years’ predictions vs. actual results to evaluate the systems. Most sites already do this. So yes, you can pull numbers out of a hat “and call it a projection”, but it’s not a particularly robust prediction, and upon future retrospective analyses it would be shown to be worthless.
Prediction systems DO use past performance as a future indicator. They just do it in a more sophisticated manner.
PS – is there any feature available where I can get an email if someone replies to one of my comments? I find it hard to remember to return to continue the conversation.
Cool stuff, Jack — any chance you could plot the BtB rankings on the same chart? I’m just curious.
I totally agree with this article. I find power rankings in general absurd, though they don’t necessarily have to be useless.
I recall seeing the Padres as a top 3 or 4 team around the second or third week of the season. I thought to myself: how can they possibly put them there? Do you really consider them the third or fourth strongest team out there?
These major companies know better than to base their rankings on the past week’s performance–it’s practically the fantasy industry’s motto (which obviously CBS and ESPN are heavily involved in).
Power rankings could be interesting if they reflected a team’s chances (from this point forward) to make the playoffs–the ultimate goal of the regular season anyway.
Interesting point. But then you would really be mixing in league and especially divisional effects, which (as stated in the article) are something that Power Rankings are generally trying to flatten.
BPro and coolstandings.com have playoff odds updated daily, in case you’re not familiar. BPro even does it two ways: assumes a team’s true talent is along the lines of how well they’ve played so far this year or assumes a team’s true talent is heavily influenced by pre-season projections (I prefer the latter).
Hear, hear. I think each power ranking should have as its intro a word about its goal or methodology. Is it aiming to be predictive about W-L records over the remainder of the season? Indicative of chances of making the playoffs or winning the World Series? Making a statement regarding the likelihood of team x beating team y in a significant sample of games?
I find it interesting (though not particularly telling) that the Angels range from 6th (FanHouse) to 17th (CBS Sports).
what is the standard of “success?” ideally you’d want your rankings today to be correlated with future wins, right? how do the “simple” rankings discussed in this piece do by that standard?
If a power ranking is trying to predict future success, you really just want an updated projection.
There are many things a power ranking could be trying to do. I’m ok with a lot of them. I just get annoyed when they try to do more than one thing (it all just gets jumbled) or they simply align with Wins and Losses (like so many media power rankings, do.)
Nobody has mentioned it yet, so I will.
BaseballProspectus just runs a formula to grab their power rankings. Although their projection systems are more proprietary than most, they are upfront about how they rank teams.
Oh and they generally keep AL power rankings separate from NL.
Yeah, Jay Jaffe’s ‘Hit List’ columns each week are a system where “power is actually ranked.” Individual successes, failures, and pertinent injuries are used to make interesting points about the status of the teams in addition to the rankings. I’m somewhat surprised a major periodical hasn’t added the Hit List to their Sunday editions for a consistent “stathead” angle.
BPro’s rankings are nice because they’re objective, I agree. Mixing wins with pythag wins with WARP2 with WARP3 just seems like a pointless mish-mash, to me, however. Maybe I’m missing something.
In my opinion, by far the best way to produce a power ranking is to simply use Vegas lines. They know as well as anyone how good teams are, and tracking lines over the course of a season can provide a very accurate snapshot of both how good a team is right now, and how good we should expect them to continue to be.
I’ve actually been doing this all season myself and I’ve been extremely satisfied with the rankings it has produced. Also, it’s much simpler than trying to produce a projection, since you’re only tracking one thing.
Do you post those anywhere, degen? Mind shooting me an email: skyking162@gmail.com — sounds like a really cool project.
Yeah, I really like this idea.
I’ve sent some explantion to Sky, but for others interested here are what I’d call the current Vegas ratings for all teams.
NL
STL .571
PHI .561
ATL .540
LAD .532
COL .520
SFG .519
CHC .511
CIN .505
NYM .500
FLA .499
MIL .496
WAS .486
SDP .485
ARI .469
HOU .433
PIT .421
AL
NYY .615
TBR .582
BOS .572
TEX .551
LAA .538
MIN .535
DET .527
CHW .512
OAK .509
SEA .503
TOR .486
KCR .456
CLE .450
BAL .422
The problem with Vegas lines is that they are influenced by perception. If Phillies-Braves is a true talent even matchup, but more people think the Phillies are better, then the Vegas oddsmakers will make the payout smaller for the Phils than the Braves, and pocket the difference- even if they know they are evenly matched. That will make it appear that Vegas also thinks the Phils are better.
It could be that wisdom of the crowd cancels this out, but I doubt it does. It seems likely that there’s a fair amount of structural bias involved (bias towards favorite teams, thus favoring larger markets, bias towards franchises with recent or frequent success, etc.).
On the 25th “Who You Face Matters” was posted by Pat Andriola. His premise was that the statistics of pitchers should be adjusted for the skill of opposing competition. Maybe a similar adjustment could be applied to team performance and power rankings. What part of team performance can be attributed to favorable or tough pitching matchups? I’m not sure that games against Halladay or Jimenez necessarily evens out during the season. It would be a nice project to include pitching matchups in a strength of schedule measure to make a better RPI for baseball.