You’re grading quarterbacks all wrong

Anyone who has read the drivel I write knows that I have been fighting a (low-stakes, metaphorical) war for some years now. Until now, this has consisted mostly of minor skirmishes and short conflicts. However, after many years of preparing, the time has come to join battle in earnest.

It’s time for the war on box score analysis.

First, let me clarify what box score analysis is and what it is not. It is NOT and and all use of numbers to try and quantify a player’s performance. Numbers can be valuable if used correctly and within context. This is particularly true over long periods of time, where numbers can be a bit more trustworthy. It is also not using numbers to quantify team production.

No, when I talk about box score analysis, I mean conversations like this one:

Fan Number 1: QB X played horrible yesterday.

Fan Number 2: Really? I thought he looked pretty good. Remember that play where-

Fan Number 1: He went 19/30 for 200 yards and no touchdowns.

Fan Number 2: Yeah, but didn’t the receiver drop that-

Fan Number 1: C’mon, don’t make excuses, he didn’t get it done. Threw a pick, too.

Fan Number 2: But didn’t that pick bounce off the receiver’s-

Fan Number 1: Numbers don’t lie, man. he was bad.

THAT is box score analysis. The use of (for lack of a better term) fantasy football stats (or other very basic stats) without context to determine how a player performed in an individual game, over the course of multiple games, or even the course of an entire season. This idea that we can, by looking at one or two numbers that contain zero nuance, accurately describe what a player did or didn’t do.

That type of analysis isn’t limited to quarterbacks. Running backs, tight ends, wide receivers, defensive linemen, edge rushers, linebackers, corners, and safeties are all subject to the same kind of conversations. Swap out “touchdowns” with “tackles” and you could be talking about an inside linebacker, you follow me?

I’ve been railing against that kind of mentality for years, and it’s time to go all in. So today we begin a series on why traditional, basic box score analysis is inherently flawed, going position by position and covering the following:

1. Why box score analysis is flawed and what you miss when you use it to gauge a player.
2. A few better “stats” that you can use than the basic, traditional ones.
3. A few things you can watch for during a broadcast to try and get a better handle on a player (I know most people don’t have time to review film for hours, I swear!).

I started here, on the Chief in the North.

Because of the nature of the medium, the depth of the analysis provided on the Chief in the North podcast (which, if you like what I do here, you ought to try out) is deeper than what I can do in written medium. But for each podcast I do, I want to give a brief summary of the points in an article for those of you who hate podcasts or who love visual mediums.

So today the war on box score analysis begins, starting with the moneymaker position: quarterback.

Generally speaking, when people gauge quarterbacks using box score analysis, they use yards, touchdowns, interceptions, QB rating, and MAYBE completion percentage and yards per attempt (if they’re going “deeper”).

First, let’s talk about what you miss. Here are two “big plays” statistically.

For either of these plays the impact on the box score is enormous. A quarterback completing a pass for 60+ yards and a touchdown has a huge impact on yards per attempt, TD’s (duh), and quarterback rating. It also obviously helps completion percentage. In the case of both of these plays, in the box score they go down as a “great” play for the quarterback statistically.

However, neither play is a great play. The first is an incredibly easy throw that travels barely 10 yards in the air and isn’t particularly well-placed. Every quarterback in the NFL can make that throw 99 times out of 100.

The second is worse, in that it’s an actively poor throw that falls short to the point that the safety is able to undercut it for what should be a fairly simple interception.

However, in both cases, because of circumstances completely outside the QB’s control (Travis Kelce’s freakishness and great blocking in one play, a terrible drop and a lucky bounce in the other), both plays instead glow on the box score as a “great play” by the quarterback.

Let’s compare these plays to another play.

On the box score, this is a wildly inferior play to the two plays above. It hurts the QB’s completion percentage and YPA, doesn’t add anything to yards/TD’s, and harms QB rating because of it.

However, this is a vastly superior play to the first two, and that’s quite obvious.

And herein lies the first problem with box score analysis: you cannot see what actually happened on the play, both in the QB’s control and the things he does not control. All you see are the results, which don’t tell you at all what the QB himself did on the play.

The idea with box score analysis is that OVER TIME, every QB will have similar luck and surrounding circumstances and so those things even out. However, it takes at the very least a full season for the numbers to be high enough for us to assume luck “evens out” (and even then, I’d argue that a few hundred plays isn’t enough to count on that), and the reality is that each QB’s situation is wildly different in terms of offensive system, overall coaching and supporting cast. In short, the box score fails to tells us whether the result was due to the QB or something else, which makes analysis on a game-by-game basis (or even over multiple games, until you’re talking multiple seasons) impossible to do accurately.

However, that’s not even the biggest issue with box score analysis (not knowing how the other players affected the play). The biggest problem is that the box score doesn’t tell what the individual actually did. I’m going to present you 2 different plays that are nearly identical on the box score.

The box score for each play is 1/1 for 25-26 yards, but those plays are night and day. One required a dumpoff when a blitz was obviously coming, an easy throw, and nothing else. The other required an incredible job shaking off rushers, keeping eyes up the field for an eternity, and absurd ball placement.

Like I said: context. The box score tells you “what” but doesn’t tell you “how.” And if you’re trying to analyze individual players, the “how” is all that matters.

Again, a 25-ish yard gain is had here. Identical to the screen and Alex’s freakish play. The box score does not tell you about...

• Pressure handled and pocket presence
• Eyes down the field going through reads quickly
• Using eyes and body language to draw away defenders
• Ball placement exactly where it needs to be

That context is everything when it comes to telling you what a quarterback did right, and you miss it by looking at “1/1, 24 yards.” A single-game box score is made up of scores of plays like this.

A final example is the holy grail of box score misses: interception numbers.

People gauge quarterbacks by interceptions more than almost any other number. Here, the pass was ruled incomplete (perhaps wrongly, to be honest), but it shows immediately the problem in conjunction to Alex’s “potential pick turned TD” play above.

Mahomes delivers a nice pass on the run here, but his receiver lets it bounce off his hands and right into the waiting arms of a defender. On paper, that’s an interception, a killed QB rating, and a bunch of people saying “threw a pick, rough game.”

All of these things are only some of what you miss when you rely on box score analysis. I go into more detail in the podcast regarding specifics on the issue but want to move on.

So what CAN we look at? If TD’s, yards, QB rating, etc. aren’t reliable, is there anything that is? Well, Mark Schofield (a fantastic QB analyst that you should definitely follow on Twitter if you do, in fact, Twitter) dropped some knowledge on me for the Chief in the North. In addition to some other great things to say, he recommended people use “adjusted net yards per attempt,” the formula of which you can find here.

I would echo Mark’s recommendation this stat, because it takes multiple things into context and attempts to paint an overall picture. Of course, there are limitations to it. But it’s significantly more informative and complete than individual fantasy football stats are.

I also advise using multiple stats to try and account for as much as you can. I would look at adjusted completion percentage (accounting for drops/throwaways) and air yards per attempt (which gives an idea where on the field the QB is going) in conjunction with ANY/A to try and paint a complete picture. Having information is good provided you properly contextualize it.

But what about what is on film? Well, the reality is most people aren’t going to waste their time doing what I do, and I can’t blame you! So if you’re looking for quick things to gauge in a quarterback while watching the broadcast, there are a few things you can check out without firing up the all-22.

• Pocket presence- is the quarterback naturally evading pressure with his movement or is he creating it? Try and imagine what the pocket would look like had the QB moved elsewhere if that helps. QB’s don’t need to be mobile to have great pocket presence (see Tom Brady sliding around the pocket).
• Eyes- Does the quarterback keep his eyes up or drop them quickly? Also, is he using his eyes to fool the secondary or are they anticipating him?
• Speed of reads- How quickly does the quarterback go from one “read” to the next?
• Accuracy- Or “ball placement.” This was Mark Schofield’s big one to watch. The difference in where the ball is placed can be the difference between a completion and incomplete pass, or the difference between a big gain or a short one. See where the ball arrives, and try to figure out where it was in relation to the coverage and when it arrived on the route (things you can often see in the replays).

There’s a lot more to be said, and I did say a lot more in the podcast about other issues with box scoring, stats to look for, and things to find on film, but I think that’s a good opening shot on box score analysis. For me, the more information and the more accurate information I have, the more I find I enjoy the game. I hope it works that way for you too. It took me years of trial and error to start at least sort of understanding football because of my stubbornness with box score analysis. Hopefully, this helps you guys skip a year or two of that.