This time of year everybody is having fun with guessing how well the Chiefs will do next year. While this discussion is fun to have, it's necessary to keep in mind a very important fact. When dealing with anything statistical, like how many teams the Chiefs will beat next season, you must consider the margin of error. For every prediction, there should be error bars. But not much, if anything, is said on how big these error bars should be. If you predicted an 8-win season, and the Chiefs only won 7, would you consider your guess to be pretty good? What if they only won 6 games? Or 5? Well, luckily for you, I have done all the hard work to get a very good estimate of how much error there should be.
This is actually a somewhat hard problem to solve for the simple reason that nobody knows how good a team really is without looking at number of wins. But number of wins is also the thing that varies. In other words, we're asking if a team is as good as an 8-win team, how many games will they win?
There are a couple ways to answer this question, but the way I chose was to create a season simulator and see how many wins I got (or, more specifically, how many times I got a certain number of wins). So first I estimated the probability that the Chiefs will win each game. Here's what I came up with:
vs. SD- .5
@ Cle- .6
vs. SF- .4
@ Ind- .2
@ Hou- .3
vs. Jac- .7
vs. Buf- .75
@ Oak- .5
@ Den- .5
vs. Ari- .5
@ Sea- .5
vs. Den- .5
@ SD- .4
@ St.L- .75
vs. Ten- .4
vs. Oak- .5
(.5= 50% chance of Chiefs win)
Keep in mind that I just made these up, that they aren't anything other than educated guesses. But, as some of you may have realized, the expected number of wins in this case is exactly 8 and, yes, I did it on purpose. With these probabilities, my program simulates a whole bunch of seasons, 100,000 to be exact, and sees how many times we win 8 games, how many times we win 9 games, etc. Here's what I got:
Interesting, isn't it? What was that? You want to know what the numbers mean? Ok then. For now you can ignore the two columns on the right. The first column on the left simply tells you what row you're on so you don't have to count. The second column is how many times we won each game. So, for instance, we won the MNF game 50,191 times out of 100,000 times (keep in mind I just made up these numbers so it doesn't necessarily mean we would really win that many times). Note how there's already some variance in the numbers. We should have won that game 50,000 times by chance, so we over-performed that game. But, since I did so many trials, they are relatively close to what you'd expect by chance.
The third and fourth column are where it gets interesting. These tell us how many times they won that many games and the percentage of times they won that many games, respectively. So we went winless once, undefeated once, and won 8 games 20,631 times. This is the important part. Despite the fact that I made this an 8-win team, they actually won 8 games only about one time in five. Although it was the most likely outcome, they were almost twice as like to have a winning season as they were to have a .500 season.
Now, here's the most important point. If you wanted to predict the win total with about 90% accuracy, you'd have to include an error of two wins according to my simulation. That means that if you, like me, think the Chiefs are about an 8 win team, you should say they'll win between 6 and 10 games. To put that in perspective, the last place Bills won 6 games last year, while the Division champion Patriots won 10 games. That means, to an average team, pure chance could be the difference between a division crown or mediocrity.
So, after all that, should we be happy or sad that there's so much error? Well, it depends on your priorities. If your philosophy is that regular season records don't matter, only championships matter, then this is very good news. I don't think anybody believes the Chiefs are the best team in the league right now, but it certainly very possible, although unlikely, to fluke their way to a Super Bowl Championship. But if you have given up on Super Bowl and Playoff dreams and are only hoping for significant improvement, this may be bad news. According to my simulation, there's about a one in five chance that they will win 6 or fewer games next year, in essence getting unlucky.
I really want to talk more about this, but the other things I want to talk about are somewhat more technical (though not too bad), and this post is already pretty long. I'll just say that if you have any questions about the accuracy of my model or want to know what the two right-hand columns are, I'll be glad to answer your questions in full, glorious detail.