The Consensus Board: Forecasters vs Evaluators

The Consensus Board can identify polarizing players and judge individual boards, but it can categorize boards. What does that tell us about the 2024 NFL Draft? What are Forecasters and Evaluators?

Photo by Jim McIsaac/Getty Images

The Consensus Big Board now includes data from 91 different analysts, which gives us a great survey from draft media on who the best players are in the NFL Draft. The ranking gives us a good idea about where the players generally slot in the class, but there’s so much more to learn about the draft from this exercise than just where players rank after a weighted average.

While we’ve covered which players are the most polarizing across all the analysts, which reveals its own insights, but there’s other ways to look at player disagreements.

In my experience collating these boards, there have been broadly two “types” of big boards. The shorthand I’ve used is “forecaster” versus “evaluator” boards, though like many things the distinctions are a bit arbitrary and there’s always been a fair degree of evaluating done by “forecasters” and a good amount of “forecasting” done by “evaluators”

Wait, What’s the Difference Between Forecasters and Evaluators?

Primarily, Evaluators are draft analysts who rely on publicly available information to craft their rankings and produce their analysis. That can include media reports of player behavior and publicly disclosed injury knowledge but largely means using broadcast footage and increasingly available college All-22 film to break down tape. They also use events like the NFL Combine to produce context for athleticism and any off-field information.

By contrast, “Forecasters” have the ability not just to use all of that prior information but access to information typically reserved for NFL teams. That comes not just from sources inside front offices and agents but also through a network of relationships with scouts and college coaches across the country. That means access not just to injury information and character data but quite possibly tests like the psychological tests teams conduct, cognitive tests like the S2 and other physical tests, like conditioning and VO2 max scores.

That might mean they have a better handle on which players might “unexpectedly” slide in the draft, but they’re also sensitive to smokescreens from their connections. They have to make judgment calls about what information is genuinely important and what might be a red herring and they’re not always right.

The history of Consensus Big Boards has shown this distinction to be valid. Boards tagged as Forecaster boards have a very tight spread and there’s a low variance in the rankings of players across that group year over year. Typically, when a player slides in the draft, Forecaster boards have that player lower than Evaluator boards.

What Happens When They Disagree?

The Evaluator and Forecaster boards can have some pretty stark disagreements. What’s strange is that their disagreements result in significant changes in pick slot – that is, where players actually get picked – but not in actual performance.