Stacking the Box Score: Week 8 against the Packers

Welcome to this week’s Stacking the Box Score, where I’ll be quantifying what went down between the Kansas City Chiefs and their most recent opponent — in this case, the Green Bay Packers, who defeated the Chiefs 31-24 on Sunday night.

If you’ve been reading this series, you’re already familiar with the metrics I’ve been using. But if terms like Expected Points Added (EPA), Average Depth of Target (ADOT), Win Probability or Success Rate are foreign to you, I’ve created a reference for all of them. You can read it by clicking here. I’ll continue to add new terms and statistical categories to the reference as they are introduced in my articles — and provide links back to the reference when they are used.

So here’s a new one: Completion Percentage Over Expectation (CPOE).

I’ve referenced something similar to CPOE in previous articles — when I have cited a proprietary statistic calculated by Next Gen Stats. That version of the metric utilizes their proprietary tracking data to estimate the likelihood a given pass is completed at the time it is thrown, averages all of those expected percentages, and then subtracts that from the actual completion percentage of a player for each game.

Hence, completion percentage over expectation.

But Next Gen Stats’ data only goes back a few years — and the public has access to their data only on a week-by-week basis, rather than play-by-play. That makes it impossible for us to find the expected completion percentage on a specific play.

But in an article on the popular data/statistics journalism site FiveThirtyEight.com, writer and analyst Josh Hermsmeyer showed that by just using the depth of target of a pass and its location (left, middle, and right — which is available publicly in the play-by-play data set), we can pretty accurately predict whether a particular pass will be completed.

Hermsmeyer argued that this is the best metric for predicting which quarterbacks will succeed in the NFL. Since then, economist and writer Ben Baldwin has popularized using the metric to evaluate quarterback performance — even finding that the metric using only publicly available play-by-play data correlates very strongly (r^2=0.88, for those curious) with the Next Gen Stats metric that uses their tracking data.

So Hermsmeyer’s CPOE metric is a very valuable tool that can show whether a quarterback is performing well in a given game — or even on a given pass.

Given that we’re debuting a new metric, this week’s article will focus largely on passing.

---

Chiefs vs. Green Bay Packers, October 27

---

Win probability

What really sticks out here is that while the LeSean McCoy’s fumble was a big swing in win probability, the Chiefs still managed to claw their way back to almost a 50% chance of victory shortly after. But any hope for a win was largely squashed when Green Bay quarterback Aaron Rodgers found Aaron Jones for a screen pass that Jones took 67 yards to the end zone — and with it, 22% of the Chiefs’ win probability.

Quarterbacks

How did Matt Moore perform when he started in place of Patrick Mahomes? We can look at how his average depth of target (ADOT) was distributed when compared to the first seven weeks of the season.

As expected, Chiefs head coach Andy Reid schemed up far more passes around the line of scrimmage and in the short area of the field (5-10 yards) than in previous weeks — when the Chiefs had Patrick Mahomes executing the scheme. How did this influence the efficiency of the passing offense and Moore’s completion percentage?

First, let’s look at the Chiefs’ quarterback efficiency using expected points added per play for each week.

While Moore certainly doesn’t have the ceiling of Mahomes, he performed far better than most (but not this analyst!) predicted last week. How did his completion percentage over expectation (CPOE) look when compared to the rest of the league in Week 8?

Moore’s CPOE was actually a bit below average against the Packers — but it was still right around zero. This means he completed about as many passes as the average QB would be expected to complete. The rest of the league, however, was more accurate than normal in Week 8. Aaron Rodgers was very accurate; given where he threw the ball, his completion percentage was 13 points higher than you’d expect.

However, both quarterbacks have nearly identical expected points added (EPA). In fact, Moore beat out Rodgers — although just barely. How surprising is that?

It turns out to be not that surprising. Moore has actually been an above-average quarterback throughout his career — and the floor for a quarterback in an Andy Reid offense is very high; when the quarterback can throw a pitch one yard forward and have a player like Mecole Hardman take it 30 yards to the house, that does wonders for a quarterback’s EPA per dropback.

Chiefs Passing Defense

Since we’re focusing on passing, I decided to break down exactly where the Chiefs passing offense has been attacked through the season’s first eight weeks, how accurate opposition passing attacks have been and how that compares to the rest of the league — and to the Chiefs passing defense last season.

These charts come from code that obtains the pass location data from the Next Gen Stats passing charts. The code was originally created by Sarah Mallepalle, a former statistics student at Carnegie Mellon University who now works for the Baltimore Ravens as a statistical analyst. Thanks to Sarah for making this data more accessible!

Interestingly, the Chiefs defense has actually been targeted less frequently in the intermediate areas of the field (and in the seams) than the rest of the NFL. On the other hand, they’ve seen far more screen passes. How successful have these passes been, and where have they been the most successful?

Well, Chiefs opponents have been more successful than average... uh, everywhere. There’s not a hint of purple (or even white) on this graph, meaning that at every location on the field, the Chiefs have allowed a higher completion percentage than the average NFL team. Not great!

But... there is a silver lining here: the Chiefs allowed an even higher completion percentage (relative to the average NFL team) in 2018, meaning their pass defense has improved.

Where has the improvement happened?

When comparing the 2019 Chiefs defense to 2018 (both relative to the league average in the respective seasons), we can see that with the exception of the deep left middle (where the Chiefs have allowed 1 of 1 pass in 2019) and the short center of the field, the Chiefs pass defense has actually done a better job at preventing opposing quarterbacks from completing passes this season. This may explain why the team opted not to pursue (or perhaps more accurately, overspend) on additional secondary help prior to this year’s trade deadline.

Decision-making

I’d be remiss if I didn’t express my views (and more importantly, the data) with regard to a few play-calling decisions on Sunday.

This tweet is from two weeks ago — after the Chiefs lost a close game to the Houston Texans. It has remained relevant, though, as the Chiefs continue to inexplicably call running plays on second-and-long — despite data clearly demonstrating the ineffectiveness of those plays in that situation.

When we look at play-calling tendencies against the Packers, the Chiefs only passed on 63% of neutral early-down situations. That is 20% lower than their number of passes in these situations in the past few weeks — when Mahomes was at the helm. Unfortunately, this confirms a worry I’ve had for a while: the Chiefs are not passing more often because Andy Reid believes passing is more efficient than rushing. Instead, he’s doing it because he believes in Patrick Mahomes.

But here’s the thing: passing isn’t just more efficient than running if your team has an amazing quarterback. Ben Baldwin — whom I mentioned earlier — keeps track of the passing and rushing efficiency for all 32 NFL teams each week. Eight weeks into the season, there are only two teams better at running the ball than passing — the New York Giants and Carolina Panthers. That means all 30 of the other NFL teams would benefit from passing the ball more often.

If you don’t believe in advanced metrics like efficiency — preferring instead to rely on stats like yards-per-carry or yards-per-passing-attempt, here’s something nine-time All-Pro offensive lineman Joe Thomas said about it in his great Reddit Ask Me Anything thread:

“Rushing efficiency is one of my favorite stats, because it tells the story of how good your run game was on [any] given day. Even average yards-per-carry doesn’t tell you as good of a story, because if you get a big lead and you’re trying to kill the clock at the end of the game, [a] one or two-yard rush might be the goal of the play in order to get a first down or to kill o’clock — so rushing efficiency tells you how often you reached the goal on any given play.”

Lastly, the play that caused Chiefs fans everywhere (or at least on my Twitter feed) to erupt in anger: the fourth-and-3 punt.

First of all, this wasn’t the first (or only) bad punt the Chiefs had in this game. On their first drive, down 0-7, they punted on fourth-and-inches at their own 34-yard line. Since 2009, fourth-and-1 attempts have been completed about 65% of the time, so attempting a conversion in that situation would have been positive expected-value play.

But expected value can be a complicated concept to grasp. Pro Football Focus data scientist Timo Riske (@PFF_Moo on Twitter) laid out the logic in a more straightforward way in this brilliant Twitter thread:

I highly suggest you read the whole thread.

But to summarize: it isn’t as simple as punting is a good outcome — and failing at a conversion is a bad outcome.

Even when a team decides to punt, there are two possible outcomes: the defense holds the other team to a stop and the offense gets the ball back, or the other team marches down the field — right back to (or past) the spot where the offense originally punted. So any consideration of whether a not a team should go for it should not just consider the likelihood of the conversion, but must also consider how likely it is the other team will return to the same spot on the field.

What Riske found is that if a team punts from their own 40-yard line, there’s about a 40% chance that the opposing team will return to that field position again. That’s a greater chance than the Chiefs had at failing the fourth-and-1 attempt: about 35%.

So punting the ball — seen as the conservative, safer option — was actually more likely to put the Chiefs in a bad position than going for the fourth down; that’s even ignoring the huge benefit of converting the fourth down.

But remember: those percentages assume an average punt and an average defense — neither of which the Chiefs have had lately; that’s precisely why the fourth-and-3 punt in the fourth quarter was so egregious.

The Chiefs were saying that they felt their team had a better chance of turning the punt into a good outcome — getting a stop and getting the ball back with time to score — than converting on fourth down. This isn’t as straightforward a choice, as teams have only converted around 50% of fourth-and-3 plays over the past decade.

However... since 2013 — when Andy Reid joined the Chiefs — the team has converted 71% of their fourth-and-3 plays. With one of the best Chiefs’ offenses of all time (even, , as we saw earlier, with Moore at the helm) against an opposing team with a very efficient offense, with the Chiefs defense playing well below average, in a game no one expected them to win anyway... you’re telling me the Chiefs felt the numbers favored them punting the ball in this situation?

Of course not — because they didn’t rely on numbers.

No... in the year 2019, when NFL teams have access to more data than ever before, when there’s a plethora of economists, data scientists and nerds posting great research online for free, when you’ve got people predicting how many yards a runner will gain just from a snapshot of tracking data... the decision that determined the outcome of the game relied on a feeling.

That’s it for Week 8 of Stacking the Box Score.

---

Leave a comment or reach out on Twitter (@ChiefsAnalytics) if you have questions or would like to see something new for next week. Thanks!