Week 3 True Power Rankings: factoring in the numbers, Chiefs’ rank stays consistent

We’ll start the inaugural edition of Arrowhead Pride True Power Rankings with a caveat:

For a few weeks, take what the number-crunchers say with a big grain of salt.

Why? Because there’s a strong argument to not even show you this information until about four weeks into the season. Until then, some of it will be a hot mess!

But we decided to do it this way because it will help give you a greater understanding of how the different rating systems we’re using actually work. Each one of them has strengths and weaknesses, and those will be on full display in the early weeks of the season. Stick with me here, and you’ll see what I mean.

Here are the four rating systems we’ll be using:

• Elo ratings published by FiveThirtyEight.com. This system uses points scored for and against as its base data and takes the strength of the opponent into consideration. Elo (named for its inventor Arpad Elo) was originally developed to rate chess players. It is the only system we’re using that — like a pundit — considers how good a team was in the previous season, and is also the only one that cares about the results of individual games; when a team wins a game, its Elo rating always rises, and when it loses, the Elo rating always falls. Read the full methodology or an entertaining FAQ.
• DVOA ratings published by FootballOutsiders.com. DVOA (for Defense-Adjusted Value Over Average) compares the result of every play in every game — taking into consideration opponent strength and game situation — to the league average for that set of circumstances, and then compiles this data into an overall percentage that shows how one team compares to the rest. This is the only system we’re using that does not use points scored in a given game as its base data. Read their Methods to our madness page.
• SRS published by Pro-Football-Reference.com. Like Elo, SRS (for Simple Rating System) uses points scored (for and against) as its base data, and considers the strength of the opponent. Like DVOA, it is expressed as a positive (or negative) number that uses the league average as its baseline. Read an explanation.
• PE, or Expected Wins. PE (for Pythagoren Expectation) was originally developed for baseball by statistical guru Bill James. It is the simplest system we’re using, using points scored (for and against) as its base data, but without considering the strength of the opponent. It is expressed as a number that represents the number of games a team should expect to win in a season. Read an explanation.

None of these systems are perfect, of course. But if you read the previous four paragraphs, you have seen that these systems are, to a large extent, complementary. Each system has one feature that is different from all the others, and other aspects that are the same as most of the rest. Three consider opponent strength, and one does not. Three don’t care which team wins or loses a given game, and one does. Three are based on points scored, and one isn’t. Three don’t consider the previous season, and one does.

See where we’re heading here? By averaging the rankings each system generates, we can balance out the advantages and disadvantages of each one, and come up with a “cruncher” ranking that is entirely data-based, which we can then compare to the opinion-based pundit rankings we gave you on Tuesday.

Additional notes for the Week 2 cruncher rankings: Through Week 4, DVOA does not take opponent strength into consideration; Football Outsiders would rather wait until a few games have been played before including opponent strength in their calculations, when it will be more clear which teams are good or bad. Pro-Football-Reference is concerned about that, too, so they didn’t publish SRS data after the Week 1 games, publishing it only after the Week 2 games were in the record books. Elo — the other system that cares about opponent strength — didn’t have to worry about this, since they can rely on the previous year’s information at the beginning of the season.

Ready? Here we go!

First, the good news: the Chiefs are looking good, with an average cruncher ranking of fourth — tied with the Jacksonville Jaguars — after two weeks.

Based on last season, Elo knows the Chiefs ought to be a good team this year, and has beaten two teams that ought to be pretty good, too. So just like the pundits on Tuesday, they’re pretty impressed. (Elo rankings tend to resemble pundit rankings more than any other data-based rankings).

DVOA has been impressed with the play of the team through two weeks — remember that DVOA concerns itself with the success or failure of individual plays — and PE likes that the Chiefs can not only score plenty of points, but so far, more of them than the teams they’ve played.

But why the SRS ranking of only 14th? We have Patrick Mahomes! It’s not fair!

It’s because SRS starts with margin of victory — that is, the difference between the points you’ve scored and the points you’ve given up. The Pittsburgh Steelers had a tie in their opening game, and lost to the Chiefs by five. That gives them a MoV of -5. The Los Angeles Chargers lost to the Chiefs by 10 in Week 1, and then defeated the Bills by 11. That gives them a MoV of only 1. So the way SRS looks at it, the Chiefs have a MoV of 15, but did it against teams that are — as far as it knows — just average.

We see the differences between these systems for other teams all over this table.

For example...

Right now, PE loves the Baltimore Ravens — very likely more than they deserve — because it only cares about points scored for and against; MoV matters a lot to PE, too. Defeating the Buffalo Bills 47-3 in Week 1, PE ranked the Ravens first that week and had the Washington Redskins (with their 24-6 defeat of the Arizona Cardinals) ranked second. But the following week, PE dropped the Redskins to ninth after they fell to the Indianapolis Colts 21-9, but only dropped the Ravens to second following their 34-23 loss to the Cincinnati Bengals.

SRS has the Ravens and Redskins at second and 17th respectively in their initial Week 2 rankings — for reasons similar to what we saw with the Chiefs. But DVOA now likes the play of the Ravens less — ranking them sixth this week, after having them second in Week 1 — and was fairly unimpressed with the play of the Redskins in both weeks, ranking them 15th after Week 1, and 17th after Week 2.

Meanwhile, Elo gave proper credit to the Ravens for the win in Week 1, but because it knows about last season, understood not to get too excited just yet; the Ravens might not be that good. Elo moved them from 11th (where they were at the end of 2017) to eighth after Week 1, but then back down to 13th after Week 2. Elo did about the same with the Redskins, moving them from last season’s rank of 22nd to 15th after their Week 1 win, but back to 22nd after their Week 2 loss.

So you might ask... since it is so simple, why use even use PE? Well, remember that PE was originally designed for baseball — where you play more teams over a longer season. It needs more than a couple of week’s worth of games to provide good results. In fact, a 16-game schedule is barely enough. But PE can sometimes see things coming that other systems cannot, just because it is so simple.

Likewise, you might wonder why we shouldn’t just depend on Elo, since it seems to have just about everything. (And it is my personal favorite for that reason). But while Elo is especially nice at the beginning of a short season — when it’s hard to tell if teams are really good, or have just beaten a bunch of teams that turn out to be poor — sometimes it can be too slow-moving.

If you read our introduction to True Power Rankings a couple of weeks ago, you saw what the crunchers said going into the final week of the 2017 season, when I completed my test run of this concept. It all made sense, because each rating system had a whole season’s worth of data with which to work. And while there will be differences between what these systems will say at the end of the season, the picture they present will be a lot more consistent than it is now.

In the meantime, because we’re averaging their results, we can still get a pretty good check against the pundit power rankings we showed you on Tuesday. So let’s get to that, shall we?

Not a lot to explain here. We’re just comparing (and averaging) the aggregated pundit power rankings from Tuesday with the cruncher power rankings from the first table.

As you can see, the Chiefs are ranked third in the Week 3 combined rankings, and there’s little disagreement between the pundits and crunchers on that score - or, for that matter, on many other teams in the top tier.

There is some disagreement, though. You may recall that on Tuesday, the pundits were abandoning the Houston Texans like rats from a sinking ship. Perhaps they saw something in the Texans that wasn’t really there — anyway, that’s what the crunchers have thought for the last two weeks.

The other big disagreement is over the Green Bay Packers and New York Jets. The pundits like the Packers a lot, but the crunchers aren’t convinced. Likewise, the crunchers see something in the Jets that is — so far — invisible to the pundits.

At this point, it’s hard to know if these are real trends that one group or the other isn’t seeing, or if it’s just the small sample size skewing the cruncher data. As the data gets more reliable, we’ll probably see the crunchers identify some teams that are rising (or falling) before the pundits do. Or vice versa.

And now to complete our weekly data dive, let’s look at the grades for the week.

All we’re doing here is converting the raw numbers that the crunchers provide us, and converting them to standard deviations from average — a statistician’s trick to determine if two values are statistically significant from each other. We express them here as letter grades, simply because they’re much easier for you to read and interpret.

Remember the rule... unless the difference between the grades of two teams in the same column is a full letter grade or more, the difference between them is measurable, but not statistically significant.

It’s your weekly reminder that bragging rights always come to the team ranked number one, but only grades can reliably predict the fall.

Enjoy the Chiefs beating the San Francisco 49ers, and we’ll see how that result — and those from the rest of the games — changes things next Tuesday on Arrowhead Pride!