Quantifying the great John Dorsey-Brett Veach debate

The idea

A recent tweet from a Kansas City Chiefs media member caught my eye. Using a list of players the Chiefs had drafted in the last several seasons, the tweet was making a point about whether John Dorsey or Brett Veach were better at finding young talent.

Two things struck me about the tweet.

• First, it was anecdotal. It was basically, “Look, only two of the guys GM X drafted were any good; the rest were bums.” That hardly qualifies as analysis.
• Second, it completely ignored the ability each general manager had at identifying young talent outside of the draft and finding quality free agents.

And of course, it suffered from the usual problem faced when trying to evaluate how well a GM acquires talent: we try to figure it out before there is enough information to make any judgment.

But it got me wondering about ways we could measure these kinds of things.

So I hit upon the idea of using Pro-Football-Reference’s Approximate Value metric as a means to answer these kinds of questions.

The methodology

Right from the start, let’s make this clear: the AV stat is a pretty rough figure (it has the word approximate right there in its name), so we have to be careful about drawing too many conclusions from it. But it can be used for all kinds of players — even players like offensive linemen for which there are few stats — and it’s data-based, rather than anecdotal.

It also occurred to me that to give this comparison any value, I’d have to have a baseline. So I decided to collect data for Chiefs players going back through 1989 — the year Carl Peterson and Marty Shottenheimer arrived in Kansas City.

I started by using PFR to collect the yearly rosters for each one of the 30 seasons in question — that is, every player who saw playing time in each season. Since PFR’s yearly rosters don’t tell you much about players who weren’t drafted, I had to do a lot of research to fill in the holes — which team had originally signed each player and at what point in their careers each player had come to the Chiefs.

I classified the players into five groups:

• Players drafted by the Chiefs (Drft in the tables below)
• Players originally signed by the Chiefs as rookie free agents (UFA)
• Players originally signed by other teams who landed on the Chiefs roster as rookies (RFA)
• Players originally signed by other teams who joined the Chiefs after their rookie seasons (FA)
• Players originally signed by other teams that contributed significantly to their previous team before joining the Chiefs (VFA)

So the first three categories cover all rookies who joined the team in a given season. The last two cover all non-rookies who joined the team.

The difference between the last two categories essentially boils down to whether they were acquired to be depth players or starters. Players who generally had at least three seasons experience before coming to the Chiefs — and also generally had an average AV of at least four in their previous three seasons — are classified VFA.

Why an AV of four? Because after I collected the rosters for all 30 Chiefs seasons, the yearly AV for all players ranged from -1 to 22. The yearly average for all players was 3.9. So an AV of four seemed like a good number to use to designate a player who had significantly contributed to a team in a given season.

(Note: I had to make some judgments about VFA players. For example, sometimes players had a low AV in the season immediately before joining the Chiefs but an above-average AV in multiple seasons before that — usually because they had been injured and played in only a few games during their last season with their previous team. I classified such players VFA because it seemed clear they had been acquired to be starters once they were healthy.)

On the other hand, I did not classify free-agent backup quarterbacks, kickers, punters or long-snappers as VFA.

The data

We can use this data to do a lot of things — and in coming articles, we will — but for today, we’re going to focus on one thing:

How well did Brett Veach do in his first season acquiring talent for the Chiefs?

The following tables show the average AV for all players in each category only during their first season with the team. In this way, we can fairly compare Veach with all other Chiefs general managers back through 1989.

Note that in each of these tables, the columns in bold type are summaries: the total average AV for all players, the average AV of the three rookie categories and the average AV of the two free-agent categories.

At first glance, this looks like a whole lot of nothing. I’ll be honest: after collecting and collating all of this data, I thought I’d see more significant differences — particularly in the Total AV/Player column.

But as you look a little more closely, you start to see some things.

While Veach brings up the rear with drafted rookies, he has the best number with undrafted rookies. His overall number with rookies is well behind Dorsey but substantially better than Peterson. With the exception of Pioli’s outlier with VFA players, there’s not a lot of space between the four GMs concerning free-agent acquisitions, but Veach tops both categories.

Here I should note that while it seems like the rookie and free-agent totals seem wrong, they aren’t. It’s just that the proportions between each rookie and free-agent category vary between GMs. Rather than averaging the averages, we’re adding up the total AV of the players and dividing by the total number of players; it’s just how the math works out.

But here’s where it gets interesting. Here’s the same data, but this time, it’s broken out not just by general manager, but also by head coach.

We tend to think that general managers operate on their own concerning personnel decisions, but this data shows that the head coaches matter, too. Peterson did much better with free agents while Dick Vermeil was coaching but did much worse in the draft with Vermeil.

(Note: This might be a data point to help confirm my long-held belief that Vermeil was simply reluctant to put rookies on the field, but by itself, it isn’t conclusive).

Scott Pioli did all right with Haley but was a disaster in his single season with Romeo Crennel.

And it also brings up another form of the question we started asking ourselves after a couple of seasons with Pioli as GM in Kansas City: was New England’s personnel success more about Pioli or Bill Belichick? If it turns out that Veach remains in his role beyond the end of Reid’s career in Kansas City, will we see something similar?

Now that we’ve come this far, we might as well break out the data by year for your reference. I’ve included additional header rows to help you keep track of the columns on mobile devices.

Conclusion

There’s no way around the fact that any early comparison between Brett Veach and John Dorsey is going to be a little fuzzy. Veach, after all, was a subordinate of Dorsey’s, and might easily share many of the same views about how NFL teams should be constructed. The picture is even more unfocused because Veach replaced Dorsey between the draft and training camp in 2017; some roster decisions from that season were made by Dorsey and others by Veach.

But what this data shows us is that both Dorsey and Veach — at least with Andy Reid as head coach — are capable of finding players who can contribute even in their first year on the team; their records with newly-acquired players compare favorably to other Chiefs general manager-head coach combinations over the last three decades.