The Finale

Well, welcome back AP readers for our fourth and final installment in the Trends N' Averages statistics extravaganza. (Which I suppose really makes this "TNA: The Climax!" (You know, I wish I'd thought of that sooner, I could have increased readership)). Anyway, if you're a dyed-in-the-wool, kool-aid guzzling homer, you may not want to read any further, because this just might be the post that turns saints_chiefsfan1979 into a believer in stats. If you know the whole debate and are familiar with advanced stats, feel free to jump right to the results. But, if you're looking for a way to avoid working while at work, or want to understand the stats a bit more, read on! I promise, no more than 4,000 words. Tops.

If you haven't followed these posts up until now, they all started with a little FanShot from JayhawksNChiefs (which you can find here) that pointed out a statistical difference between Matt Cassel and our newly acquired pigskin chucker, Alex Smith. You see, the main concern is that Matty Nice's career average numbers look similar to Alex Smith's career averages. Very similar. And while I agree that average value should be the first thing we look at statistically, it is not the only thing. Two more facets of a player's career are important. First is consistency (or in stat speak, the variance of our QB measurement). The second is trend data (commence derisive comments from RamX21 in 3...2...1...mark).

Now, consistency is important because it gives us an idea of how accurately we think we can project stats into the future. If a QB varies wildly from year-to-year, then it's tough to place a probability on how well they might do next year. A steady player gives us a solid idea of what to expect next season. We can actually see consistency in each of our charts below by how "wavy" or sporadic the curved lines are for each quarterback.

Thus, trends become important because if a QB is wildly inconsistent, we simply expect a regression to the norm. But, if a QB is consistently rising or falling, it is not unreasonable to expect they will continue to track that way. Or, at least, that is my assertion. Talking about trends in this way is actually very common. We all do it routinely without even realizing it. We say things like, "he's lost a step," or we talk about the "learning the game," and make statements like "In 2011, Justin Houston had zero sacks in his first 11 games, but racked up 5.5 sacks in his last 5 games. His future looks bright!" These all refer to a trend. Now, all we have done is use numbers and statistics to back statements like these up -- to see if people are actually producing better or worse.

In my follow-up to JayhawksNChiefs (which I should have called "TNA 2: More Curves!"), I simply used basic NFL QB stats to show that Alex Smith is improving in every single statistical category except "completions per game," whereas Matt Cassel has consistently gotten worse in every single statistical category (link here). I included comparisons to Tom Brady (primarily to show that Alex Smith is definitely NOT Tom Brady) and Joe Flacco (as an example of a QBs whose numbers Alex Smith could match if he continues his upward trend) and they both ALSO trended upward in all but one statistical category. This was done to show that good QBs get better, bad QBs do not, and that my mystical "trend" data supports that notion.

Now, my stats, my method, and my trends were contested or questioned in many ways. First, I was accused of ignoring the past, which is awkward seeing as how you can't have a trend without a past. Second, I was told that my stats were split incorrectly and I was therefore ignoring the fact that Alex Smith had a noodle arm. This led to my second post (Which should have been called "TNA 3: Deep Balls!") which showed that Alex throws the long ball with the same consistency and improvement as seen in the rest of his game (again, link here). Third, it was rightly pointed out that my stats ONLY examined how well Alex Smith threw the ball, and did not factor in things like sack rates, turnovers, "clutch" play, and his injury history. These are all very real, very important considerations. Which brings us to today and this post -- the advanced stats version of TNA.

Now, I was never trying to avoid these last important questions; they simply took some time to get a handle on. You see, several groups of football junkies around the intrawebz like numbers even more than I do. They have been seeking the Holy Grail of QB stats that can measure how great a QB truly is. As we all know, however, trying to isolate a QB's "talent" in the middle of a team game as complex as NFL Football is a daunting task. Yet, undeterred, many men have tried, and I looked at the best of these stats to create this post.

It took a long time to collect all this data. I would estimate it at roughly two-and-a-half metric crap-tons of time. You see, the people who make these stats for a living don't want them to be easily copied, pasted, or downloaded. In fact, sites like Football Outsiders want us to pay $40.00 for that privilege. But the information is there, you just have to click on each QB for each stat for each season individually. I collected around 1,200 of them.

My initial aspiration was to compile all of these stats into some kind of "Uber Stat" that I could use to evaluate Alex Smith. This proved beyond my capabilities. We can't just "add-up" or "average" these stats. That would be like saying, "let's combine Touchdowns, Completions, and Yards for wide receivers by adding them together and dividing by three." If we did that, the WR with 50 catches, 1500 yards, and 50 TDs would lose out to the WR with 110 catches and 1495 yards, and no TDs -- which makes no sense. We instead need to normalize and combine the stats, weighting them to make sure they tell us what we want to know. This is precisely what each of our advanced stats does. When I tried to them combine them together - I failed. This is not surprising. First, each stat is measuring a different aspect of the QB's game. Second, thousands of man hours have already been spent trying to optimize these stats, so it's not shocking that I alone could not out-think all of ESPN, Football Outsiders, Brian Burke, and the guys at Pro Football Reference combined. I was bummed out though.

So what does this mean? It means we get another post with lots of individual charts instead of just a few cool Uber Charts. But, as this is my last post on this topic, I did not want to cut corners. I wanted the most complete comparison I could effectively generate for our boy Smith -- not to support some mythical homerism, but to truly get a feel for what we have. I therefore picked the nine most comprehensive stats I know of, and divided them into three categories: how well a players executes as a QB, how well that translates into production, and how much that production contributes to his team. I will explain these stats, and the four groups of quarterbacks that I compare Alex Smith to. What we should come away with is the most complete picture of "who Alex Smith really is" that numbers can provide.

Now, I'm not a slave to numbers. I would prefer to watch football over analyzing it, but hey... it's the off-season. Film study is a very real, very important part of this process, but I think we should at least know the numbers if we want to talk about them. All I can attest to is this: these stats include everything. There is no trickery or bias. No injury, no strength of competition, no strength of defense, no fumble, no pick, no dropped ball, or receiver corps has been missed by these stats. The numbers are what they are. Their interpretation, however, is left completely up to us. I will give you mine. So, without further ado...

via showmemoweather.files.wordpress.com

The Advanced Statistics

(Click the name of the stat for a link to the host website which gives a more thorough explanation.)

The Execution Stats:

I call these "execution" stats because they try to evaluate how well a QB has actually produced on the field. They are measuring the same general thing that the basic stats in my earlier posts were measuring, which is "how good is a QB on each play that he plays" -- only these stats do it better do it better.

• Adjusted Net Yards per Attempt (Pro Football Reference): This stat, known as ANY/A+, is like looking at a QBs yards per attempt, but it includes bonuses and penalties for TDs, INTs, Fumbles, and sacks. If a QB has turnover problems, it will show up here.
• Expected Points Added per Play (Advanced NFL Stats): This stat, known as EPA/P, is a the main building block of the advanced stats movement. EPA looks at how many "expected" points a play adds to give credit for "scoring" even when a player doesn't score. The classic example is JC breaking off a 90 yard run but getting tackled at the 1, then Smith punching it in on a QB sneak. With this stats, JC gets most of the credit for that TD. Like ANY/A+, it takes into account turnovers and points scored, but it factors in down, distance, current game score, and time remaining in the game. So, not only does JC get more credit for that TD, but he gets bonus credit if it was late in the fourth and sealed a victory, vs. a garbage time TD when down by 40 points.
• Defense-Adjusted Value Over Average (Football Outsiders): This stat, known as DVOA, takes EPA to the next level by additionally factoring in the strength of the defense they are playing against, so a 2012 TD against the 49ers counts as more than the same TD against the Saints. It also noramlizes the stat by looking at "value over replacement," which means this stat is not measuring how good a QB is, but how much better of worse a QB is than a calculated "league average" quarterback. This is therefore the most complete execution stat, and the best execution stat for making comparisons.

The Production Stats:

The "production" stats add the dimension that was missing in our earlier posts in that they take into account how much a player is on the field. Alex may be the greatest QB ever, but if he only plays in 8 games in a season, he could grade out as average.

• Expected Points Added - Passing (ESPN): This stat takes the EPA we talked about above, and adds up the EPA a QB earned on every pass play they were involved in. As mentioned, EPA already takes into account turnovers, down/distance, score, and time, but ESPN adds in their own "clutch-weighting." Their "clutch" formula is proprietary, so I have no idea how it works, but they claim it does.
• Expected Points Added - Total (ESPN): A "clutch-weighted" total of EPA from all of a QBs plays, including Pass EPA, Rush EPA, Sack EPA, and Penalty EPA; everything really except hand-offs. This is probably the best production stat for QBs.
• Expected Points Added (Advanced NFL Stats): This stat is the cumulative total of all plays where the QB touched the ball but without ESPN's "clutch-weighting." I added it specifically so you can see the difference between the EPA per Play above, and this EPA (which both use the same formula). The difference between the two stats is our best indicator of how much Smith's injury history has affected his production.

The Contribution Stats:

The EPA stats get us halfway to the promise land, but they include the whole team's contributions on each play, and just add them up for a QB if he was involved. These "contribution" stats take the final step in trying isolate the QBs play from the rest of the team. None can claim the title of "the one stat to rule them all," but they are the best three stats available for QBs. Period.

• Win Probability Added (Advanced NFL Stats): This stat, known as WPA, is very sensitive to context. It can help us see who really mattered most when it was important. It's not just, "who helped us score," but "who helped us win." With EPA, you could see who was most important to your offense, but this shows you who was most important in the game. For example, in a tight, defensive, field position game that the Chiefs win 3-0, Colquit could end up with the highest WPA without winning the EPA. This is probably my favorite stat to see how much a QB matters to his team.
• Total Quarterback Rating (ESPN): This is the "new" QBR from ESPN and should not be confused with the old "QB Rating." QBR takes all of the EPA and WPA information, and further divides it by the QB's involvement. It takes into account things like YAC, pass protection, and whether a dropped pass was the QB's fault, the WR's fault, or was just a great play by a DB. While WPA is a better "team" stat, QBR is the better "QB" stat.
• Defense-Adjusted Yards Above Replacement (Football Outsiders): This stat, known as DYAR, takes WPA and factors in the defense teams were playing, normalizes the stat, and again compares it to the average QB replacement that's available. This means your not comparing Alex to the average NFL quarterback, but the average NFL back-up QB. This gives you a better measurement of how much value a player adds to his team and provides a larger "penalty" for missing time and games -- a very important thing to note when it comes to Smith. This is probably the best stat to assess a QB's total value.

The Comparisons

I have chosen to stick Alex Smith's stats up against four groups of other quarterbacks. Each group serves a particular purpose. Overall (hopefully), they can tell us exactly where Smith currently sits. The comparison groups are:

The Elites:

These QBs are not chosen to show that Alex Smith is elite. If anything, they show you he is not. The advantages of including them is two fold: First, I want to demonstrate that these statistics work, and seeing this comparison should help convince us of that. Second, it will show us what every NFL franchise should shoot for (even if Smith will never get there. And, begin "we need to draft a 1st round QB debate in 3...2...1...mark.) You'll notice that the less successful Manning brother is not included here. Ever since horse-head mounted up on horse-face Donco style he has been dead to me...

• Drew Brees
• Tom Brady
• Aaron Rodgers

The Winners:

Many comments on the previous TNA posts asked for this comparison. How does he stack up against the "Non-Elite" yet "Super Bowl Winning" Quarterbacks. This is perhaps the most important comparison groups because it shows us exactly what we hope Alex Smith can become - a herd working, game winning, successful post-season contributor.

• Eli Manning
• Joe Flacco
• Ben Roethlisberger

The Others:

These boys are each included for their own specific reason:

• Jay Cutler - Gives us a rare glimpse at an NFL QB that had a history with one team, got traded, and started for second team for several years. This is pretty rare in the NFL, and may help us understand how the trade could affect Smith (although its important to note that Smith has already had three head coaches and 8 different offensive coordinators already, so he's pretty used to transition)
• Matt Cassel - Some folks won't let the comparison die, so he's back. He does provide a good baseline, however. Again, Smith has played like Cassel during his career. The question is, can he separate himself from that past?
• Mark Sanchez - I included Marky Mark mainly to convince folks that these advanced stats are better than the basic QB stats like completion% and yards. As many of my stats dejectors have pointed out, Sanchez, like Alex Smith, is trending up in the basic stats. Now, Matt Cassel trended down in the basics, so we expect him to trend down in the advanced stats. But, will Sanchez, who trended up, still look like he's improving, or will the stats tell a different story?

The Legacy Comparisons:

This last category also comes from some feedback, requests, and comments. I kept Jay Cutler in as a baseline. I then include two QBs from days past. I put in McNabb because some folks wanted to know "how Alex stacks up against other Reid quarterbacks." Will he be the next prodigy of the Walrus? I only include McNabb's Philadelphia playing time; I did not include his Washington season in order to preserve the "Reid" comparison. I then added Trent Green from due to many comments that say, "Alex may never be great, but if he can just manage the game like Trent did, we'll be doing well." Well, I wanted to know how Green and Smith measured up. Green is also a "traded" QB, but he doesn't have enough history with his first teams to establish a decent trend like Cutler does. There are no ESPN stats that go back far enough for either McNabb or Green, so those will be removed in these comparisons.

• Jay Cutler - Year included are 2005 to 2012. Cutler was traded from DEN to CHI between the 2008 and 2009 seasons.
• Donovan McNabb - Years included are 2002 to 2009. McNabb played in PHI in all eight seasons.
• Trent Green - Years included are 1998 to 2006. Green played 15 games for Washington in 1998. He did not play in 1999 and that "gap" year has been removed (this will alter his trend line). In 2000 Green played 8 games for STL. Green started all 16 games for KC from 2001 to 2005, but his concussion cut his 2006 season short to 8 games.

(Note: I don't normally like comparing QBs this way - across different years. The NFL is cliquish and streaky - some seasons are the "year of the quarterback" and others seem to favor rushing. I can't explain why, but you see similar dips and rises in all QB play from season to season (just look at the "elite's" common rise from 2010 to 2011 and common dip back down in 2012 in the charts). But, this is the best I can do, and it provides a decent side-by-side look -- just take it for what it's worth.)

The Results

(right-click or command-click on the charts to see them larger)

Elites - Execution

Again, in order, these stats show how well a QB plays, how much he adds per play, and how he compares in those ares against the "average QB. First observation: Smith does not yet play elite football. Second observation: Smith does have elite consistency. His upward trend over the last four seasons (not just the Harbaugh years) shows very little fluctuation. This gives us hope that his play can overcome the trade to KC and perhaps avoid a large fall off.

Elites - Production

So, Smith may play at a high level, but he doesn't produce anywhere near elite level football. Elite QBs are adding 60 more "points" per season to their teams than Smith is.

Elites - Contribution

Smith's total contribution to the team tell a similar story. Again, these stats are, respectively: how much he matters, how well he plays, and his total value. You'll notice his WPA is just above zero, even in his best years. This is the definition of the "game manager." He neither wins, nor loses football games. Also, we need to beware of Smith's ESPN QBR rating. Remember the consistency we raved about above? It's not true with his QBR. His 2012 season is what we call an "outlier," meaning it unfairly skews his trend data. That line should NOT point up as high as it does, and we need to keep that in mind when we look at QBR in the following comparisons.

Winners - Execution

As our previous posts showed, Alex started to play like a winner, especially in his last two years. This could be the "Harbaugh Effect," but again, his consistency is tight. An argument can be made for either side, but the FACT is he's there right now. Can he stay there?

Winners - Production

Boom. This is the slide that tells the biggest part of the story. Smith's production, when factoring in games played and staying on the field, just isn't at the level of our other SB winning QBs. Even in 2011, when Smith played his best ball for a full 16 games, only his Passing EPA was up at their levels. This is not a great sign.

Winners - Contribution

What do these pictures show us? First, SB winning QB are NOT game managers. They add wins. Flacco's steep trend in WPA is a good indicator of his ability to pull out big wins. Again, we need to be wary of Smith's sharp ascent in QBR. The only warm fuzzy we get is that DYAR, Smith's total value for a team, is right at Flacco's level now.

Others - Execution

Again, as Aiken_Drum loves to remind us, Smith's averages are right on top of Cassel and Sanchez. The best differentiator is slope. Of note, we see that in all of the advanced stats, Sanchez is now trending down (see, these number things might work.) We should also look at consistency. Cassel's lines are everywhere whereas Smith has steadily pulled away. Unfortunately, however, look at Cutler's plummit after his trade in 2008. Could this mean a similar dip for Alex?

Others - Production

Again, Smith is pulling clear of the Cassel / Sanchez basement as is producing right in the Cutler wheel house.

Others - Contribution

Here, Sanchez's true colors ring out. He is awful. Yet, we also see that Alex is right at Cassel levels in WPA. We absolutely NEED Smith's trend to continue or all of his great QB play may not elevate the team. As we've said, Alex probably won't lose us games, but can he win us games when we need them? Not according to these trends and numbers.

Legacy - Execution

What do our legacy comparisons show us? First, we see how much trades affected Green. Once established in KC, it took him a full season to settle in, and then -- wow, he produced at very high levels. Second, at his very best, Alex has just now only caught up to Donovan McNabb's effectiveness on the filed. Alex has never produced like Green did. Are our "Green" expectations too high? Likely yes. Can Reid make Smith his next McNabb? Perhaps.

Legacy - Contribution

Looking at the WPA, the first thing we see is that Trent Green was much more than a game manager. From 2002 to 2005, he was elite. Second, Alex has never added points or added wins like the others.

Conclusion

Well, I hope - if nothing else - that this post gave you some knowledge and confidence in the advanced NFL stats. I know it's a long post, but it can serve as a solid reference if you want to get deeper into these stats - even if you don't care about the Alex Smith investigation. As for my conclusions:

I stand by my assertion that in terms of basic QB play, Alex Smith will perform on par with the likes of Joe Flacco and Big Ben next season. These numbers do not suggest he will win like them. Can our running attack compensate for this? Can Smith keep his upward trend going and buck the dip that hit Cutler and Green with their trades? Most importantly, will he stay on the field or will we have to Chase our way to the offseason? (Now, I'm not a big pun guy, but heck, I'm proud of that one).

All-in-all, the advanced stats sit right around the over-unders we have seen in recent posts. According to our numbers, Alex should fall about 5 wins below the elites, two or three below the winners, one below Cutler, and two above Sanchez. Smith and the Chiefs hit an average over/under of 7.5 wins. The other over-unders?

NE 11 Wins (+3.5), GB 10 Wins (+2.5), NO 9 Wins (+1.5), NYG 9 Wins (+1.5), PIT 9 Wins (+1.5), BAL 8.5 Wins (+1), CHI 8.5 Wins (+1), NYJ 6.5 Wins (-1)

If anything, our stats have their BAL and NO totals too low, and the Jets too high - a fair assessment.

So, is Smith the answer? Well, as we all know - only time will tell. My best prediction? Smith will play well, much better than Cassel and in the top 15 in the league. This may not translate into wins. To do so, JC will have to continue his greatness and our Defense and Special Teams will need to be solid. Smith, even if he continues his solid upward trends, is probably not the franchise QB we have all been dreaming of. He is, however, a welcome respite from the last four years. The Chiefs are still waiting for their true first round quarterback.