I. We Need a Quarterback
This article is about Quarterbacks. I know we need one, you know we need one, and most everyone else does too. The exception may be the draft pundits, who are mocking DTs to KC as though we haven't drank from that well recently. I want to emphasize that not only do we need a QB, we need an elite one. I'll make the argument here that I make anytime I talk about QBs. If you expect to have a realistic chance to win a Superbowl going into the season, year to year, you must have an elite QB. It's not an option, but a prerequisite. With an otherwise elite team around him, and excellent coaching, a QB who is merely very good can do it. That's as low as you can go and still have a shot at a championship, save for some anomalous circumstance. To slam this point home, review the last decade of football. How many average QBs, or the merely good ones have won Superbowls? The answer is zero. You have to go back to 2003 ('02 season) to find a case of an average QB winning a Superbowl. Trent Dilfer's Buccaneers defeated Rich Gannon's Raiders in a game Oakland handed away. If you weren't paying attention to football then, you should read about it. It was certainly an anomalous circumstance. This season just ended with Joe Flacco leading a team to a championship. He's a pretty good QB. I could put him in the top 10, but not much higher in my opinion, at least not yet. Flacco is as "bad" of a QB as I think can get a team a Lombardi trophy without being an aberration . For the most part though, we see the same teams in the playoffs year to year, and the same ones making Superbowl runs. That is no coincidence. They're all teams with top-of-the-line QBs at the helm. If you have an elite QB you have a legitimate shot at a title. If not, you're hoping minority statistical probability blesses you. I'd prefer the former, so I'm about getting the best possible QB in KC. To that end I've spent the last two months, mostly over the inter-semester break, doing research and statistical modeling to see if one of those elite QBs is available in this year's draft. This is going to be long and wonky, so bear with. If not, glance at the pretty colors, then shoot to the summary section for the TL;DR. I put a lot of time and effort into this though, so I hope your eyes can soldier through it.
This section will be especially nerdy, so I'll give you a pass if you want to skip it. It's juicy though, I promise. My goal was to build a model that could uncover an elite QB using basic statistics. It's moneyballing, if you will. To do that I worked backwards from the statistics of current elite NFL QBs while they were in college (toatalfootballstats.com). My suspicion going in was that their statistical production would probably be similar enough to create a definable in-group. If not, the project would not be worth doing. Initially it appeared that I was wrong; the numbers varied too much to make projections from. The most problematic stat was completion percentage. I kept going back and looking to see if the numbers were truly random, or if there was a meaningful variation. It turns out the variation was very meaningful. There is clear and definite inflation in the completion percentage numbers. In the Elway generation, a future elite QB completed 55% or so of his passes. By the Manning/Brady/Brees generation is was just over 60%. By the Roethlisberger/Rodgers generation it was mid 60s. I needed to both prove the trend was real, and gauge its magnitude. For every current Superbowl champion QB, I plotted the completion % for their ultimate and penultimate years in college. The longitudinal period was from 1996-2007. For years in which I did not have data (2001) I used dummy smoothing variables. All that does is keep the interval constant (1yr.) without upsetting any trend that may emerge. Then I ran a regression to determine strength of relationship and year-to-year rate of change. The time correlation increase was heavy (r=.57) and the rate of change pretty steady. Basically a good college QB should increase their completion percentage by about a third of a percentage point each year, all else being equal. For any wonks: (y =.39x + 60.5 | x = yr - 1996). This does two more cool things other than just reveal an inflation pattern. First it lets me determine an "optimal" completion percentage. Second, it allows to to adjust that number so that my model can look at players in the future.
For the other metric categories I use to grade QBs, I again researched future Superbowl champion QBs to determine statistical ranges. I determined a theoretical point of perfection, that is a 100% score, and an average, that is a 75% score. from there any number could be thrown in and divided by the range to give a score. There is one other important facet to my model, which is that a QB can score TOO high. QBs do not continually increase their grade by increasing their statistics. If you score over the projected elite range, you're downgraded. It's sort of of like writing 1000 words on a 700 word paper; your professor docks you for making them read it, when you should know ain't nobody got time for that. To manage the downgrade, I installed a dummy grade above "A" that acts as essentially a backside to the "A" so as to put the numbers I've calculated as optimal atop a bell curve, rather than as a zenith. I do this because the optimal point will often be in the middle of an optimal range, though not always(eg. wonderlic). I will submit that my method probably goes a little soft of downgrading over-performance. You have to get way over to drop below an "A" grade. So the effect is mostly to tame numbers down to reasonable expectations. Maybe that's valuable; maybe I'm being soft. Either way I don't think it did anything to any of the QBs I analyze that appears unjust or out of hand.
So my process has been: 1. Determine relevant factors to grade on QB performance 2. calculate a research-based optimal grade and range for those categories; 3. compile and grade the statistics of a subset of QBs in this draft class. The relevant factors I grade are completion percentage, win percentage, avg. yards per attempt, TD;INT ratio, and my consistency metric(more later). I grade the numbers by plugging it in to a formula which tells me where it lands in the range, with deductions for over-performance. Once I have a grade, for example, 77% of the optimal score, I grade it--C+ in this case--and then I figure an overall grade for each QB based on a GPA calculation of their line scores. I use the final two seasons for each QB for their stat set. The final season is weighted higher than the penultimate, as I assume it's a better projection of NFL readiness, and credit should be given to progression. In this case all the major QB prospects actually took a hit for regression as Seniors.
III. Metric Analysis
For this exercise I selected five QBs from the 2013 class to grade. The QBs are Geno Smith, Tyler Wilson, Matt Barkley, Ryan Nassib, and Mike Glennon. Smith, Wilson, and Barkley I chose because of interest and the fact that they've got the biggest names. Glennon was rumored to be a Pioli favorite when I started this, so included him as well. Nassib was someone I looked at once I had the model taking shape as I guy I thought could grade out as a mid round steal. You'll get to see if I was right about that or not. Before I get into the metrics I did grade, I want to discuss two that I think are incredibly overblown.
Arm Strength. This seems to be all anyone talks about with QBs anymore. Arm strength works into my grading scheme as an element of completion percentage and YPA avg. But as it's own indicator, it's the most overrated measure there is. The biggest arms in the league belong to Matt Stafford and Joe Flacco. Flacco now has a Superbowl ring, but look at the other guys who do. They don't posses elite arms. Rodgers is pretty good, but guys like the brothers Manning and Tom Brady are quite average. Drew Brees is below average I'd say. It doesn't matter if you can throw a football through a brick wall. I care about throwing a 15 yard out route well. You need functional arm strength for that, which is not nearly as sexy as what scouts want to see. Consider that the average NFL throw is something like 11 yards. Strength is just one element of the overall ability to deliver a ball quickly, and in the right place. Read speed, release pattern, and anticipation all figure in. A big arm is not a big deal. Sure, it's nice, but it's not something to get hung up on, especially if they play well. It's also the easiest thing to improve. I can get your arm strength up with an NFL conditioning program. I can't make you any smarter.
Size. Being tall is certainly a benefit as it provides better field vision. But plenty of QBs play well without having to rely on it. At 6'0 Drew Brees has done okay, I'd day. Aaron Rodgers, a measly 6'2, can't keep himself out of the playoffs. When evaluating size, you should consider how that QB plays at and with their size, not set a categorical absolute standard. For plenty of scouts it's 6'3 or bust. The modal height for an NFL QB is 6'2 and plenty have and do succeed at a high level at that height or under.
Being 6'5 with a nuclear arm is good, but it's bonus point. QB is a position played with the mind, so I value it much, much more. But not that I've disposed of what I don't much care about, let's more on to the ones I do
A. Completion %
Here is the grade chart for each of the 5 QBs on accuracy, which is derived from completion percentage only.
You see clear class delineation here. Mr. Smith and Mr. Barkley are peers on this metric, though they've taken different routes to get there. Barkley was worse as a senior, but still quite good. The weighting hurt him, as did the over-performance adjustment in his junior season. He gets an A-, and only barely at that, but I think it's an honest appraisal, especially relative to Mr. Smith. Geno had a big jump in completion % as a senior. He went from very good to ridiculous. The model priced in some skepticism of those numbers, which brought him down to a solid A. The model says both are very accurate passers, but that Mr. Smith is somewhat better. I'd say that squares well with the tape. Wilson and Nassib are draftable on this metric, but only barely. Glennon was terrible. His F as a Senior makes him undraftable as far as I go.
This is an assessment of basic avg. YPA. I could tell it mattered from my research, but it's impact seemed to be less. if you're over 7YPA, you're probably fine. As such it only weights 1/3 as much as the other metrics. Failing this metric should worry you, but beyond that it shouldn't push a prospect's stock too far one way or the other.
No one grades out poorly. I would like to point out though, that Mike "the cannon" Glennon does the worst. Great arm strength doesn't mean you have the best ability to get the ball down the field. If everything else checked out fine, It would not bother me, but it's only one of a few warts of his. For all the complaining about Mr. Barkley's arm strength, he looks just fine here, enjoying the company of known slingers Wilson and Smith. All check out, thus we move on.
Again, a fairly straightforward metric in the TD to INT ratio. What you look for here is that the player exceeds two TDs for every one INT thrown. Even that is actually a bit toward the low end anymore. 3 to 1 is a nice mark to pass. It wasn't often that guys got past 3 to 1 but it didn't seem to be a negative indicator when they do. A fairly high INT total wasn't so bad of a statistic so long as a proportionally high number of TD come with. Interestingly, 12 INTs is a common number for future elites to throw.
Again Mr. Smith totaled some silly numbers in this category, but wasn't punished this time around. I suspect too high a spread on this might hint at something worth questioning, but the research didn't warrant it. Mr. Barkley's grade took a modest blow for his higher INT total as a senior, but it wasn't devastating. Mr. Nassib was more even Steven on this than the others. Mr. Wilson and Mr. Glennon round out the bottom with numbers that didn't impress me, though they weren't bad enough to make big dent on their score. I was kind of surprised how soft you can be with your margin on this, so much so that I questioned it's utility at first. But this is another one of the marks that will alert you of someone who isn't operating as a basic level of proficiency. Baline Gabbert would have failed here miserably, and yet he went high in the first.
I care deeply about a QBs ability to win games. The position has the greatest impact on the outcome of a game. So then, if you're a good QB, you should have a good record. Defenses should struggle against you, not the other way around. This metric is just a calculation based on percentage of games won. As with all of these, the final season weighs most. I figure that season's stats as 150% the value of a junior season. 20 wins is a common standard QBs are graded by, but I personally find this to be too simple. What of someone who wins 20 games, but loses 20 games as well? That impresses me far less than someone who wins 17 games, but played only 20. Sample size is an issue, but 2 seasons suffices for me.
That's a lot of red. Everyone but Mr. Nassib had a worse senior season which is painful given my weighting. Mr. Smith and Mr. Wilson were hurt most by their underwhelming finishes. Mr. Wilson's grade was absolutely destroyed by it. He went from a perfect score as a junior to failing the metric with his awful record as a senior (scored a 42%). Mr. Smith and Mr. Barkley both won the same amount of games through their last two seasons, and the same number in each season actually. What helped Barkley was that he played fewer games due to bowl ineligibility and injury. You could argue Mr. Barkley is rewarded for taking the field less, but that forces us to play counter-factual scenarios which are just unknowable. My next metric has something to say on the matter, however.
What's this? I don't know if "Consistency" is the best name for it. Maybe you could say it's clutch or something, so permit me to explain. I wanted a way to measure blame for a loss. Like I said earlier, good QBs should defeat opponents, but thing's can and do happen. What would a brilliant QB on a bad team look like? It was an especially important question this year given that the whole crowd had a bad senior season. Maybe West Virginia was destined to suck this year due to transferring into a tougher conference, despite Mr. Smith's efforts. Maybe the chaos in Arkansas overwhelmed a good solitary performance on the part of Mr. Wilson. Or maybe Mr. Barkley's team fell down on him in the clutch. You would rather just not have to try to find the least bad excuse, but this year we're forced to.
Here is what I have done. I figured the average points scored for each QB in their games won, and for their games lost. The greater the difference between the two, the more blame I assign the QB for the loss. I begin with the average points scored in a win, with the grade being calculated as a % of that number the offense scores in a loss. A slight curve is applied to the loss scores, mostly because you can't expect two separate sample sizes to make a dead on statistical match. If there is near parity, one can assume the QB put up a win caliber performance regardless of actual outcome. What the model is actually doing is blaming the offense or the defense for losses. Therefore if WRs dropped every pass thrown to them, the model would miss that. However if a QB brought his team back for the win, but the defense let the opponent in the end zone one more time as time expired, the offense would be treated as a winner. As the QB has out-sized impact on the performance of the offense, the credit or blame flows to him. One last thought on possible flaws in this. White this didn't come close to happening with this population, a player could take a massive hit for only one bad game. Say a QB has a 12-1 season, but lost that one game in a shutout. They would fail the metric for that. I'm considering a redesign where this is tied to the win propensity metric, and scaled based on number of losses. That is to say the more a QB loses, the more the model would check up on them for fault. Just some thinking out loud.
Matt Barkley was steady Eddie. Win or lose, he posted identical point totals. That says to me that, generally speaking, he doesn't shoulder much blame for USC losses. Mr. Glennon's score here was confounding in a couple of ways. As a junior he did poorly on this metric. But as a senior, he averaged a higher point total in losses than he did wins. That gets him a perfect score but I'd be lying if I told you I know what to make of that. It is also odd to me that he did so given how bad he did in the rest of the metric categories. Finally we find Mr. Smith's wart. He didn't have many losses as a junior, but as a senior he racked up plenty. It would seem that he was at least as much at fault for their losses as any other factor. Notably, as a senior he had the second worst spread disparity of any season in the sample (English: when WVA lost , he sucked). Only Tyler Wilson's junior season was worse, but that may have been a bit of over punishment for only a couple of losses. Probably a fault in the system there, but I don't feel too bad about it here because he continued to stink it up in losses the next season. It may be a case corrective error, but the final score looks congruent with the body of work.
I cannot complete this metric because the class has not yet taken the wonderlic test. Once scores are out, I will factor it in and grades will be complete. I'll also do a post later on with some analysis on the wonderlic. Suffice it for now that the test matters, and matters big time. You may have read that wonderlic scores don't matter. It's a common meme I couldn't disagree with more. More sophisticated looks that have dismissed the wonderlic suffer from two pretty basic flaws. Again, I'll go through it in a post once we've got leaked scores. They could tank the guys doing well in my model, or it could crown a clear cut winner.
IV. The Role of Subjective Analysis
I'm a big fan of statistical analysis. If you can find the right set of lenses to look at them through, I think you can see more direct answers to complex problems. My boy Nate Silver has been doing it for a while now in electoral politics. Of course in football there's the old expression "numbers don't tell the whole story." I do agree with this, but I also believe they can tell you most of it, so long as you know how to read that story. I don't know that I know exactly how to read that story. Nate Silver is a better statistical modeler that I will ever be and he submits that football probably can't be moneyballed like baseball has been. I've just been motivated to try because the team I love is probably about to make an investment decision that will make or break the next decade.
So whatever the percentage of the story stats will tell you, you're left with subjective analysis to fill in the balance. Take numbers over hunches, but if the tape gives you good reason to reject them, believe your eyes. I reviewed at least some game film of every QB in this draft. I watched every snap of Mr. Barkley and Mr. Smith's senior season, and most of their junior seasons. I'm left in basic agreement with what my model says. Now there are players other than the ones I've analyzed here. I didn't run his numbers, but I could tell by glance that E.J. Manuel would score very well. Tape tells me there may be some funny business afoot. Granted, I didn't take the intense look I did at Smith and Barkley, but it didn't strike me as NFL serious football play. If Manuel crushes the wonderlic I'll assess his numbers in full.
I've compiled all the data into line scores, and those into overall scores which are charted below.
Mr. Barkley has graded out, at least provisionally, as the best QB in this class. While he doesn't get an elite grade, it's not far off. Mr. Smith grades out as someone who can be good starter. Everyone else is looking like fringe starters or backups. So then it comes down to a battle of Smith vs Barkley. Subjectively I like Barkley better by about the same margin the model does. I've heard nonstop about Barkley's awful senior season. It may have been underwhelming relative to national championship expectations, but I just don't see it in his performance. Furthermore, the other two QBs most frequently mentioned going before him in the draft did have disastrous senior campaigns. Mr. Wilson was dreadful. It amazes me he's getting a pass on such a catastrophic season while Barkley is being pilloried for a what was still above average. I suspect it has to do with his better arm strength and magic inch (6'3!). The same goes for Mr. Smith but he was certainly better than Mr. Wilson. There is a lot to like about him. I judge him as easily the best thrower of the football in this class. His passes have great zip and the placement is incredible. Mr. Smith's accuracy is his most impressive attribute. He hits tight spots to undersized receivers all day. Ball security is another big plus as his interception totals show. He does get afforded ample space to make throws quite often though. That just will not be the case in the NFL. His biggest flaw is in his head, which worries me much more than Mr. Barkley's flaws. Smith will post video game numbers against easy competition, then struggle against more even matches.That tells me he could struggle with the higher talent level in the NFL. I also question his mental toughness, which is the biggest red flag for me. When the going gets tough he seems almost confused that he's not just winning. Against K State he fell apart mentally. His irritation was obvious in the bowl game vs Syracuse. That isn't a bad thing on it's own. Peyton Manning does it often, but he does something about it. Mr. Smith tended to go on to defeat. Conversely some of Mr. Barkley's best games were losses. I was impressed with him in losses to Oregon, Arizona, and UCLA. Barkley clearly posses good mental toughness. He will get into a brawl and go blow for blow for you. If he loses it's because he lost to the clock, or the defense is leaky. Vs UCLA he brawled back, overcoming early mistakes on his part, to put the game within grasping distance. Unfortunately he got his shoulder mangled before he could complete the comeback. The UCLA defense wasn't much help either. He has a quiet aggressiveness I like. Some of his interceptions as a senior appeared to be a result of him getting over aggressive and push passing. His ball placement is quite good (not Smith good), and he knows it. His confidence got him intercepted when he had to turn the heat up in the middle of a brawl. Arizona State wasn't a very good game for Barkley. He played sloppily, yet was still able to overcome it to put his opponent away with two scores to spare.
If the NFL draft were held today, with the first overall pick I would select Matt Barkley, and feel comfortable doing so. The talk is now that he could go in the second round. While there is legitimate reason to prefer Smith over Barkley, anything beyond the first is silly. Nor can I find merit in drafting anyone other QB ahead of him. He's played well at the position of highest value. He's as NFL ready as it gets schematically and mechanically, so he should be competent to start day one. My expectation is that Mr. Smith will be taken by the Chiefs first overall. If this happens I won't lose any sleep over it, as he has talent and Andy Reid is a same pair of hands for him to land in. He might be a better fit in Reid's offense anyway. I do have the sneaky suspicion that Mr. Barkley will embarrass those who pass him over. That will be especially true if someone gets cute and drafts Mr. Glennon over him.
Thanks for reading.