A Realistic Evaluation

Those who read AP regularly may know that I was a Physics major and a Math minor, and as such I've gained a certain view of the world. Generally my posts are filled with stats to prove my points, as statistics can be a very powerful tool in understanding what is really happening in a certain situation.

But stats can also provide more "big picture" lessons. Even if you have no mathematical ability at all, an understanding of the logic behind statistical analysis can lead to a more rational view of the world. In this case, it can help us analyze what exactly is happening in these preseason games and what it means for the regular season.

But before we get to that, I want to look at two games we played last year, because I think they offer a lot of insight into the way football works, and are very good examples of what is going on in our preseason games.

On October 9th, 2011, the Chiefs played the Colts in Indianapolis. We were coming off our first win of the season, and hoping to continue to wipe away the memories of our first two blowout losses. Unfortunately, as the game began it seemed like we were doomed to another blowout loss. The Colts jumped out to a 17-0 lead early in the 2nd quarter, and although we managed to score a touchdown, they responded with one of their own, keeping the lead at 17 with 1:15 left in the half.

I watched this game at a bar in St. Louis where several Chiefs fans gather to watch the games every week. At this point everybody had largely given up hope and even I thought a comeback was very unlikely. But sure enough the Chiefs managed to score a touchdown with 13 seconds left in the half, then scored two more touchdowns in the second half to take the lead 28-24, which ended up being the final score.

The second game wasn't as happy for us, although it looked like it could be at first. This game was also on the road, except this time we were facing a much more daunting task. Matt Cassel had just suffered a season ending hand injury, causing what may be the worst QB to ever start a game in the history of the NFL to get his first start. To make matters worse, we were facing the New England Patriots that week.

Despite the overwhelming odds, the score remained tied for most of the first quarter until until Palko "led" a drive in which he completed one pass for seven yards that nevertheless went down the field and resulting in a field goal, putting us ahead 3-0. And the score remained that way until 5 minutes left in the half, during which the Patriots scored 10 points.

At half we were only down 7, which is far better than we had any right to be. In the second half the Patriots turned it on, scoring 24 unanswered points leaving the final score at 34-3.

Had these games been preseason games, the starters likely would have been pulled at half, leaving fans with the opposite impression of what the final result did. In the Colts game, it would have looked as though the Colts were far better, and Arrowhead Pride would have been in full panic mode. With the Pats game, many would be convinced that the Chiefs would have been the superior team if we had been healthy, considering we were down only 7 missing three Pro-Bowl players from the previous year and a very good tight end.

These conclusions would have been entirely wrong. Even if we would have been fully healthy last year, the Patriots would probably still have been much better. And even though we weren't so great without the ACL crew, we were still much better than the Colts.

How could this type of thing happen? Looking at it statistically can give us the answer.

Randomness and Small Sample Sizes

If you flipped a coin twice, how many heads would you expect to get? One, right? Since it's a 50-50 proposition, and you do it twice, it should end up heads once. Even though it's the most likely result, there is only a 50% chance that it will happen. The other 50% of the time you either get two tails or two heads.

Ok, so what happens if you flip a coin 100,000 times. This time you would expect 50,000 heads. Although the likelihood of getting exactly 50,000 is very small, the probability of getting close to that is very high. But the probability of getting all heads or all tails is basically zero. In fact, the chance of getting fewer than 49,000 heads is so low that you have a better chance of being Todd Haley than you do of getting that. That means that there is better than a 99.99999% chance that you'd get within 1% of that 50,000 you expected.

Now let's say you have a coin that you're not quite sure is fair. You flip it twice and it comes up heads both times. Would you then conclude that it isn't a fair coin? Of course not, a fair coin would come up with that result 25% of the time. You clearly need a larger sample size to come to any definite conclusion.

This principle applies to everything that has an element of randomness to it. Almost everything you do has some degree of uncertainty, but at the same time it's not completely random. This is especially true for football.

In the NFL, any team can win any given Sunday. It's not because all teams are equal, far from it. It's because there is some randomness to it, so every once in a while a bad team will beat a team that's better than they are.

Now think about this for a moment. What is more likely; an underdog is up at halftime, or an underdog winning the game? If you've been paying attention, the answer should be obvious. the first half is shorter than a whole game (duh), so the halftime score represents a smaller sample size than the full game. And, as I showed above, the smaller the sample size the more randomness rules the day.

If you pay attention to it, you'll notice that it's not really uncommon at all for underdogs to be close or even leading at halftime in any sporting event. But it's usually the case that the better team ends up winning by the expected margin in the end in these games. That's because as the game goes on the sample size becomes larger, and the randomness gets less important and the normal result dominates.

Sure, the Colts were able to hang a pretty good sized lead on us, and we were able to hang around with the Patriots, but they were due to luck. In the end all teams involved showed their true colors.

What is Repeatable

When dealing with the preseason, it is critical to keep in mind that you're dealing with very small sample sizes. When the starters play only a couple drives, simply looking at the score won't tell you very much, because that score will be heavily influenced by randomness. Even if you exclude the scores by backups and only look at what the starters did, it doesn't give you a very good picture.

That means that, when analyzing a preseason game, you have to be extremely selective in what you consider meaningful. Instead of just saying whether the team looked good or bad on a drive, you have to ask whether they looked good or bad in a repeatable way. In other words, is what just happened likely to be something we can do consistently or not?

Turnovers, for example, are very random. Teams with very high turnover differentials tend to have disappointing years the following year, because those turnovers made them look better than they really are. Another case is big plays. It is unusual to be in a place where big plays are the norm on either side of the ball. You aren't going to win games by trying to throw 40 yard passes every play.

Things that are repeatable include long drives with medium length plays and few penalties. It also helps if those plays aren't SportsCenter Top 10 plays either. These types of drives indicate an offense that can move the ball consistently. It's much harder to have a 10 play TD drive from being lucky than a 4 play one.

With this in mind, let's look at our two "bad" preseason games.

The Rams Game

In the game against the Rams both teams had their starters out for 3 drives each. Needless to say, those three drives represent a very small sample size, which means that randomness ruled the day. The Rams' first drive was 4 plays long and involved a couple big plays. This by itself would seem to be a result of luck rather than skill (unless our defense made a habit of allowing big plays). Then the Baldwin fumble can also be considered luck, as fumbles in particular are basically Roulette on a football field. (Except in the case of QB's, but that's a topic for another day.) The subsequent touchdown was a result of a very short field. We managed to score a touchdown on a long drive, and both teams had medium length drives that ended in no points.

This game was a good example of how a worse team can look good for a short time. Make no mistake, the Chiefs are much better than the Rams, and had the starters played the rest of the game we likely would have ended up ahead.

Although on the surface it looked pretty bad, in reality it only looked that way because of plays that are not repeatable, and shouldn't be much cause for concern until they show a pattern of happening often.

The Seahawks Game

This game was much more "legit" than the Rams game. When the Seattle lead was 16-7, it had been a typical game with very little random influences. It doesn't mean Seattle was necessarily better, but their lead showed that they were at least competitive with us. It is very rare for a significant underdog to have a lead like that without fluky plays.

However, the collapse after that point was very much a fluke, and due entirely to luck. the refs gave them 14 more points, which put the game away. Had that not happened, the Chiefs may have been able to have a lead by the time the starters were pulled.

In this game I felt their lead was almost entirely due to Cassel being a little off. Sure, there were some drops, but almost all of those drops were due to either inaccurate passes or good coverage. Because of this we put our defense in a tight spot, and it led to their 16-7 lead.

That 16-7 lead is actually pretty interesting. With three field goals, it could have easily been much worse had they done a little better on a couple plays and scored touchdowns instead. On the one hand, this might mean that they got kind of lucky on their drives, since our defense able to stop them short of the goal line most times. Or it could mean we got lucky that we stopped them, and it is actually worse than it seems. Someday I might investigate whether lots of field goals is a sign of a lucky defense or a lucky offense, but I digress.

And before I close, I want to point out two more things. Firstly, good teams can look bad, although it is not very likely and usually happens against other good teams. If you don't believe me, look at the Patriots' playoff game in 2009.

Secondly, and more importantly, we need to give a ton of credit to the Seahawks. Don't be fooled by the score, we have a very good football team here in Kansas City, and the fact that they were able to do so well against us without relying on fluky plays shows that they also have a pretty good team. I'm not sure I'm quite ready to say they're better than us, but they will be a very tough team to beat this year.

(As a post script, I want to explain why I'm so confident that we'll be good. As far as I'm concerned, we have the closest thing to a "Dream Team" as you're going to get. Even though we don't have the best QB in the world, he is good enough to take advantage of the incredible talent around him. Even in the Seahawks game you could tell that at points our talent was just too much for them to handle. In particular our offense will no longer have down games like we've been seeing the past couple years where we score 10 or fewer points. Even in the worst games we're bound to put a couple drives together just from sheer talent.)

(As a second post script, I want to share a concern that I'm pretty sure only I have at this point. Other than Cassel not being too good at first against Seattle, I'm very pleased with our offense. We are able to move the ball very consistently. However, our offense is largely based on shorter plays, relying largely on converting 2nd and mediums and 3rd and shorts to get first downs. This approach seems to be working, but with more plays means that it's more likely that turnovers happen. Most good offenses get yards in 15 and 20 yard chunks, limiting the number of plays they run, and thus reducing the number of turnovers. We saw a glimpse of that in the Seattle game, where a very promising drive ended in a pick 6. One mitigating factor may be that turnover are far more likely on passing plays, so our running and short pass offense will be less likely to turn it over on any given play. This is just something you should keep in mind.)

This is a FanPost and does not necessarily reflect the views of Arrowhead Pride's writers or editors. It does reflect the views of this particular fan though, which is as important as the views of Arrowhead Pride writers or editors.

Log In Sign Up

Log In Sign Up

Please choose a new SB Nation username and password

As part of the new SB Nation launch, prior users will need to choose a permanent username, along with a new password.

Your username will be used to login to SB Nation going forward.

I already have a Vox Media account!

Verify Vox Media account

Please login to your Vox Media account. This account will be linked to your previously existing Eater account.

Please choose a new SB Nation username and password

As part of the new SB Nation launch, prior MT authors will need to choose a new username and password.

Your username will be used to login to SB Nation going forward.

Forgot password?

We'll email you a reset link.

If you signed up using a 3rd party account like Facebook or Twitter, please login with it instead.

Forgot password?

Try another email?

Almost done,

By becoming a registered user, you are also agreeing to our Terms and confirming that you have read our Privacy Policy.

Join Arrowhead Pride

You must be a member of Arrowhead Pride to participate.

We have our own Community Guidelines at Arrowhead Pride. You should read them.

Join Arrowhead Pride

You must be a member of Arrowhead Pride to participate.

We have our own Community Guidelines at Arrowhead Pride. You should read them.




Choose an available username to complete sign up.

In order to provide our users with a better overall experience, we ask for more information from Facebook when using it to login so that we can learn more about our audience and provide you with the best possible experience. We do not store specific user data and the sharing of it is not required to login with Facebook.