How to improve NFL draft pick value charts

Since the Kansas City Chiefs will enter the 2022 NFL Draft with 12 selections — and we expect general manager Brett Veach will be trading some of them — we’re now examining how to improve the draft pick value charts that are typically used to evaluate trades.

In the first part of this series published last Friday, we covered the strengths and weaknesses of the commonly-used draft value models — and then examined a 2008 attempt at creating a new one based on PFR’s AV metric.

Picking up where we left off

Back in 2008, Pro Football Reference founder Doug Drinen put in the work to use his own approximate value (AV) metric to create a draft value chart. It would stand in contrast to the famous Jimmy Johnson (JJ) model (loosely based on draft pick trades from before NFL free agency began in 1993), the more-recent Rich Hill (RH) model (which in practice, turns out to be almost identical to the JJ model) and the Fitzgerald-Spielberger (FS) model that is based upon the values of NFL players’ second contracts.

As we showed on Friday, Drinen’s effort was the right idea. While his approach had some problems, it was based on what drafted players do on the field — that is, the kind of production we can reasonably expect from players selected at different points of each draft.

Historical draft pick success rates

When I started this project, I presumed that since 1999 (the last year for which Drinen collected draft data), NFL general managers had gotten better at their jobs; there would be fewer draft busts in the data.

This turned out to be (more or less) correct.

This chart shows the percentage of successful draft picks from 1989 through 2016. Here, a successful pick is defined as a player who provided the drafting team an AV of 12 or more — that is, three seasons of average play, four seasons of somewhat below-average play or a steady climb from poor play to average or above-average play.

As we see, over this 28-year period, only about one in three picks across all seven rounds of the draft meet that reasonably low standard.

Still... over this period, the trendline shows that NFL teams have been making a slow, steady improvement towards getting their draft picks right — albeit with a bit of a stumble in the early 2010s.

Of course, it’s evident that higher picks are much more likely to succeed, right? The data shows this is true — but perhaps not as much so as you might believe. Once again, defining a successful pick as one that gives the drafting team a player who accumulates an AV of 12 or more, here are the round-by-round success rates during that same period.

So historically, even a first-round pick only has about a three-in-four chance of producing a player who is a contributor. In the second round, it’s a bit over a 50/50 chance... and so on.

So the next time you’re arguing to draft a first-round player because they are an “instant game-changer,” remember that there is one chance in four you’ll be wrong.

Data parameters

Sharp-eyed readers may have already noticed that there is no eighth round in an NFL Draft. But it does consist of eight rounds’ worth of players.

The extra 32 players are those added after the third through seventh rounds as compensatory picks. The number added to the end of those rounds varies year by year, although the total number of comp picks always equals 32. In the 2022 draft, most of the sixth-round selections will actually take place after pick 192 — making them what we’ll call virtual seventh-round picks. Then, almost every one of the seventh-rounders is after the 224th pick. That makes them virtual eighth-round selections.

So unless noted otherwise in this article, any reference to a draft round will be to a virtual draft round. That will include historical picks. In our calculations, we’ll ignore the original draft round, converting it to a virtual round.

But why 1989 through 2016? Why not go back farther than that — or include data through 2021?

At the front end, we’re ignoring draft data from before free agency. Before it began in 1993, players were more likely to stay with their original teams for a longer period than they do today; to include them would make the data less relevant to current conditions. (The data begins in 1989 because players drafted at that time were the first to become eligible for free agency).

At the back end, the data stops with 2016. This way, each player in the dataset has had at least six seasons to make their mark.

Building an AV-based draft value chart

One of the unfortunate legacies of the JJ chart is that aside from the first overall pick, it has ridiculously undervalued all the early-round selections. It is based on the idea that the relative value of draft picks is logarithmic — which is to say that there is a very steep drop-off in the value of picks right from the beginning of the draft.

Even I fell victim to these legacies of the JJ chart. When I plotted Drinen’s 2008 data in charts for Friday’s introductory article, I incorrectly assumed the data’s trendline would be a logarithmic curve. I never bothered to check the trendline’s R2 value, which is used to check its faithfulness to the data. (An R2 of 1 means the curve fits the data perfectly). The logarithmic trendline I gave you for Drinen’s data on Friday had an R2 value of 0.96 — while its fifth-order polynomial trendline had an R2 of 0.99.

That doesn’t seem like much of a difference — but the trendlines (and especially the data that is then extracted from them) are substantially different.

This chart shows the difference in the two curves when applied to Drinen’s 60th percentile AV data.

Now, look what happens when the more accurate polynomial trendline is used to determine the specific values for each pick — and those values are then compared to the three draft value charts that are in use today.

Before, Drinen’s pick value data simply fell on a curve between those of the JJ and FS models. Now we see that Drinen’s AV-based data is sharply different.

The charts in Friday’s article — and some conclusions drawn from them — have now been updated. I apologize for the error.

But in a way, I’m glad I made this mistake — because it explains some things.

A few years after Drinen’s data was published, Chase Stuart also did some work on an AV-based draft value chart. It has been used as the definitive AV-based draft chart in dozens of articles you can find on the Internet.

But when we plot his chart against the commonly-used charts, it’s obvious that Stuart made the same mistake I did: he bought into the idea that draft pick values are logarithmic.

In fact, Stuart’s AV-based draft chart values line up very closely with the incorrect logarithmic trendline I applied to Drinen’s data; Stuart’s values also lie between the FS and JJ charts.

While I’m tempted to say more about some of Stuart’s methods — which I find suspect on some levels — let’s leave it at this: like my original analysis of Drinen’s data, it’s flawed on a basic level. It assumes the underlying data is logarithmic.

If I hadn’t made the same mistake, the difference between Stuart’s model and mine would have been hard to explain.

The definitive AV-based draft value chart

With all of that out of the way, the rest is relatively easy.

Because he wanted the values on his published chart to be smoothed out, Drinen averaged the AV of players taken within five selections of each pick, thereby increasing his sample size. Unfortunately, this created problems for the first and last-five picks — and made outliers affect larger potions of his data. But now, we can use spreadsheet tools to calculate an appropriate trendline — and then use it to calculate each point on our chart.

And since we don’t have to adjust our AV data because a large portion of it was before free agency began, we can simply use the Draft AV column in PFR’s draft tables, representing the AV earned by a player while with their original team.

Here’s what that looks like:

In this chart, the data points (the red dots) represent the average Draft AV for all players selected with each pick between 1989 and 2016. We can then use the black trendline to calculate the precise values for each pick.

Then adjusting those values to the 3000-point scale, here’s how it compares to the common draft pick value charts.

Other than extending past the 224th pick of the draft, the curve is very, very close to the polynomial curve we created from Drinen’s 2008 data. This suggests two things: one, that on the playing field, the NFL world hasn’t changed that much since free agency began — and two, Drinen did a pretty good job adjusting his AV data from the period before free agency.

The table of values for this AV-based draft value model is at the end of the article.

The bottom line

As noted in Friday’s introduction, the Jimmy Johnson draft chart is the gorilla in the room. As long as a significant number of NFL general managers continue to use it, it will influence draft pick trades. And while the Johnson chart remains prevalent, the Rich Hill chart (which tracks the historical value of draft picks) isn’t going to change much.

But the league’s general managers are intelligent people. I fully expect that, eventually, they’ll all come around to using this chart — or one like it. Why? Because it better represents the actual, real-world value of draft picks. Besides... eventually, an NFL GM will start using a chart like this one to take advantage of other GMs. When they realize they’re being bamboozled — and they will — they’ll have no choice but to follow suit.

All of that may take a while. In the meantime, this model will provide little value in accurately predicting draft trades; most GMs will be looking at a different one.

But starting now, this model is an excellent tool to evaluate trades that have already been made, allowing us to better estimate the real-world consequences of trades that were based on the outmoded Johnson model.