Finding a better way to value NFL draft picks

In less than two weeks, Kansas City Chiefs general manager Brett Veach will begin what many are calling the most crucial draft of his career. He will enter the 2022 NFL Draft with a dozen picks at his disposal — not to mention more than a few substantial needs on the team’s roster.

At present, most observers expect Veach will not use every one of their picks to select a player. With a well-earned reputation for aggressiveness over his first four drafts, he is expected to make trades — either before the draft to acquire a veteran player or during the draft to move up (or down) the draft board.

So now is an excellent time to take a detailed look at how draft picks are currently valued — and how the models now in place can be improved.

Current draft valuation models

There are three commonly-used draft pick valuation models. The most famous (and, as far as we know, the most frequently used) is the Jimmy Johnson chart created by the Hall of Fame head coach while he was with the Dallas Cowboys in the early 1990s — which has become so ubiquitous that it is often called “The Chart.” Another is the one developed by Rich Hill of our sister SBNation site Pats Pulpit — which is (unsurprisingly) known as “the Rich Hill chart.” Then there is “the Fitzgerald-Spielberger chart,” created by (you guessed it!) Jason Fitzgerald of OverThe Cap and Brad Spielberger of Pro Football Focus.

So let’s compare them.

Please note: the Rich Hill model is based on a 1,000-point scale, rather than the 3,000-point scale used by the other models. So for this comparison, it has been converted to a 3,000-point scale.

As we see here, the Jimmy Johnson model (hereafter “JJ”) and the Rich Hill model (“RH”) are nearly identical. That shouldn’t be a surprise, because the RH model is said to be based on historical draft trades. Since the JJ model has been widely used for more than two decades (and was said to be loosely based on historical trade data), the RH model simply confirms that the JJ chart is the current standard.

And if you ever run across someone who says, “I actually prefer to use the data-based Rich Hill chart because it’s much better than the seat-of-the-pants Jimmy Johnson chart,” you’ll now be able to tell them something they don’t (apparently) know.

We also see that the FS model is substantially different than the other two. And it should be because it’s based on an entirely different data set: salaries paid to NFL players.

What to like (and dislike) about each model

The JJ chart has one big thing going for it: everyone uses it. Whether you like it or hate it, you have to pay attention to it. As the old saying goes, it is the gorilla in the room.

The 3,000-point scale is also nice because it allows nicely-detailed values without using annoying decimal points.

But it has many problems. One is that it only runs out to the 224th pick. That made it obsolete almost immediately upon its formulation — because, in 1994, the NFL started mixing in a round’s worth of compensatory picks after the third through seventh rounds; these so-called comp picks are meant to compensate for team losses in the free-agent market.

An even bigger problem with the JJ chart is that it places virtually no value on picks after the sixth round. Pick 193 has a value of 14 points, while pick 224 is valued at just two points. We’ll all agree that players taken in the seventh round are less likely to have a significant NFL impact than those taken with the first overall pick. But to suggest that the first pick is more than 200 times more valuable than the 193rd selection (or even worse, 1,500 times more valuable than the 224th pick) simply staggers the imagination.

Why? Because according to the NCAA, more than 16,000 college football players were eligible to declare themselves for the 2019 NFL Draft. Just 254 were selected that spring. Even if you think that the talent pool is more likely to consist only of the roughly 6,500 Division I players who were available that year, this still means that the college players drafted in 2019 were all likely to be above the 95th percentile of the talent pool.

While it is way more than likely that a seventh-round pick will not “stick“ in the NFL, a handful of them do precisely that every season. As the NCAA data suggests, it’s probably not that these late-round picks don’t have the talent to play. It more likely has to do with whether they are drafted into a good situation for them — and whether they’re able to find an opportunity to get on the field. So while the pick used to select them may not be worth a lot, it is certainly worth something.

Photo by Tom Szczerbowski/Getty Images

That brings us to the FS model, which places a high value on late-round picks. This is primarily because of a built-in bias of its salary-based formula: it is based on the second contracts of players drafted between 2011 and 2015. This means that draft busts — that is, players who don’t earn a second contract after their rookie deals conclude — aren’t really represented. In addition, every NFL player is paid a minimum salary based on their experience. These two characteristics of the FS model tend to overvalue the late-round picks.

As its authors note, this does remove a lot of variance from their model. But you don’t need me to tell you that even first-round picks can (and do) become busts. Variance absolutely exists in the draft market. If a draft chart is to be accurate, it must take this into account. And it must also factor in the on-field contributions that each drafted player provides to their new team — something that none of these models consider.

Still, the FS model does give greater value to picks in the draft’s middle rounds — and shows a less extreme curve in the first 64 picks. Anecdotal evidence would suggest that these could be significant problems with the JJ chart.

Doug Drinen’s AV model

In 2008, Pro Football Reference founder Doug Drinen did some pioneering work on a draft value chart based on the approximate value (AV) metric he had created.

This statistic places a single numerical value (always a whole number) on any player’s season, at any position, from any year going back to 1950. While it reflects standout performances (quarterback Tom Brady is the career leader in AV, leading a top 10 that includes Peyton Manning, Brett Favre, Jerry Rice, Ray Lewis and Reggie White), it also credits players who get playing time in secondary roles — for example, as reserve offensive linemen or special-teams players. An AV of 4 in a given season is considered average.

AV isn’t perfect. As I’ve noted in previous posts referencing the statistic, it has the word “approximate” right there in its name. But the other word is “value” — which made it perfect for Drinen’s purposes.

He looked at the career AV of every NFL player drafted over the 20 seasons from 1980 through 1999. (His idea was that by 2008, most of the players drafted in 1999 would have finished their careers). In order to have the data more closely reflect only the AV provided to the team that drafted each player, Drinen discounted AV earned after each player’s fourth year. (It’s important to note that his dataset included 14 seasons that took place before free agency began in 1993).

To account for variance, Drinen presented his data in a multi-column table representing five tiers. I’ve charted them to give you a clear visual perspective of what they represented, adding the most representative polynomial trendlines to better represent the raw data.

The “Top 3” data is just what it says it is: the average of the top 3 players in adjusted career AV who were selected at each pick of the draft; it represents the maximum value a team could expect with a given pick. (One thing that this makes clear is that it’s possible to get elite players anywhere in the draft). Then the successive tiers broke out the variance. Drinen gave an example of a team holding the 22nd pick.

[The team] could conceivably get a 124 draft value player [the “Top 3” tier] if they get really lucky. They have about a 20% chance [the 80th percentile] of getting a 52 or better, a 40% chance [the 60th percentile] of getting a 39 or better, a 60% chance of getting a 26 or better, and an 80% chance of getting a 14 or better.

Note that Drinen was taking those numbers directly from the raw values in his tables, represented by the dashed lines in the chart. The trendline would indicate that the “really lucky” drafting team would be more likely to get a player who would give them something closer to an AV somewhere around 118 — not 124.

Drinen’s approach had other flaws. One was that in order to increase the sample size (and “smooth out” the data) for each draft pick, he averaged the data for five picks on either side of each selection. For example, the value for the 22nd pick was the average of the data from the 17th through the 27th picks. This created a problem for the first (and last) five picks of the draft.

Furthermore, in his effort to account for multiple levels of variance, Drinen made the chart difficult to comprehend. He never explained which set of data should be used to value draft picks — although the 80th or 60th percentile data is probably what he had in mind. So his idea never caught on.

Still... he was on the right track.

Plotting the curve for Drinen’s 60th-percentile data (and adjusting it for a 3000-point scale) alongside the well-known draft charts, we see that his data is substantially different. Like the FS model, it gives greater value to picks made in the second and third rounds (which is likely to be a flaw of the JJ model) but goes well past the FS model in the first round. Like the JJ model, it doesn’t extend past the 224th pick, but it does give a more realistic value to late-round picks that the JJ model does.

And Driden’s data has one big advantage: it is based only on the performance of players on the field.

---

So can we improve Drinen’s approach — and update it with data more relevant to today’s NFL? That’s what we’ll cover next week in the second part of this series — in which we’ll create an AV-based draft value chart that makes sense.