What Can the Preseason Tell Us About Rookie and 2nd-Year QBs?

We obsess over the preseason every year and then quickly forget about it once real games start. But is there value in it? What about for QBs like J.J. McCarthy who didn't get a rookie year?

Photo by Adam Bettcher/Getty Images

The return of preseason football has, every year, been celebrated as a “return to football,” a celebration that quickly dies down once the game actually starts. Journalists and diehard fans will keep preseason games on through the final whistle, and even to those dedicated students of the game, bad performances are waved away as merely the quirks of preseason play.

After all, aren’t you supposed to be testing the limits of your capabilities? If you don’t make mistakes, how can you learn what the risk-reward calculus for your play is? And what good is evaluating a player when they’re playing with backups?

We do not, conversely, treat good performances as worrisome; they just prove that the team is ready to play. After all, they may be playing against backups, but they’re only supported by backups!

The preseason is an absolutely wonderful playground for confirmation bias.

It’s at least worth considering if we can learn anything from the preseason, especially as we enter the regular season with two first-round quarterbacks from the 2024 draft class without significant (or any) time in the regular season in J.J. McCarthy and Michael Penix Jr.

Given this particular lack of information, it’s reasonable to cling to the preseason to figure out what the regular season might look like for rookies and second-year players without much playing time. Can the history of preseason quarterback performances tell us anything?

What Do the Data Say?

Early-Season Returns

Anecdotally, we recall a few poor preseasons for quarterbacks that ultimately led to outstanding rookie or second-year performances. C.J. Stroud’s rookie preseason was mediocre – 4.5 yards per attempt, 1-1 touchdown-interception ratio and a PFF grade at a replacement level of 61.1.

He went on to win Offensive Rookie of the Year, with good reason.

On the other hand, Dak Prescott and Baker Mayfield turned their excellent preseason rookie performances into outstanding rookie years. Any model built off a sample of about 30 dropbacks is going to feature high levels of uncertainty. After all, projecting a rookie season based on the first two regular season games is going to be pretty difficult.

What would that even look like?

It looks about as random as you’d expect. In fact, it looks slightly more random than I would have guessed, given that the grades from Weeks 1 and 2 are 1/8th of the season-long grade.

This at least suggests something we’d expect – not only are small samples unstable, but inexperienced quarterbacks are, too. Because most of them fall under the line where season-long grade matches the grades generated from Weeks 1 and 2, most of the instability is generally positive. Put another way: rookies improve over their initial showing.

The average Week 1 and 2 grade for rookies who played significant snaps throughout the season and early in the season is about 55.5. The average season-long grade was 64.5. Jayden Daniels’ grade improved from 53.7 to 89.6!

And of course, it would be against type for me to ignore that Carson Wentz’s rookie grade dropped from 90.0 to 69.9.

Nevertheless, we might be able to construct models based on the information available to us to project season-long performance for rookie and second-year quarterbacks.

Draft Capital

Any model we construct will have to beat a fairly simple approach: using where a player was drafted to figure out how they’ll play in their first or second year in the NFL. If we incorporate preseason data into our model, and it makes the model less predictive than draft position, then we functionally learn no information.

What does draft pick position tell us about rookie-year performance? Let’s translate draft pick into an “expected” PFF grade and measure that against the outcome we actually see.

This appears worse visually, but it has a stronger relationship (measured by correlation) with outcomes. In this case, it means that the average distance between a point and its expected spot on the line is lower than it is with the early-season returns.

Though it’s not remarkably predictive (a correlation coefficient of 0.31, which is higher than the coefficient of 0.20 from early-season returns).

Preseason Rookie Grade