Photo: USA Today Sports

Welcome to Regression Alert, your weekly guide to using regression to predict the future with uncanny accuracy.

For those who are new to the feature, here's the deal: every week, I dive into the topic of regression to the mean. Sometimes I'll explain what it really is, why you hear so much about it, and how you can harness its power for yourself. Sometimes I'll give some practical examples of regression at work.

In weeks where I'm giving practical examples, I will select a metric to focus on. I'll rank all players in the league according to that metric, and separate the top players into Group A and the bottom players into Group B. I will verify that the players in Group A have outscored the players in Group B to that point in the season. And then I will predict that, by the magic of regression, Group B will outscore Group A going forward.

Crucially, I don't get to pick my samples (other than choosing which metric to focus on). If the metric I'm focusing on is touchdown rate, and Christian McCaffrey is one of the high outliers in touchdown rate, then Christian McCaffrey goes into Group A and may the fantasy gods show mercy on my predictions.

Most importantly, because predictions mean nothing without accountability, I track the results of my predictions over the course of the season and highlight when they prove correct and also when they prove incorrect. Here's a list of my predictions from 2020 and their final results. Here's the same list from 2019 and their final results, here's the list from 2018, and here's the list from 2017. Over four seasons, I have made 30 specific predictions and 24 of them have proven correct, a hit rate of 80%.

The Scorecard

In Week 2, I broke down what regression to the mean really is, what causes it, how we can benefit from it, and what the guiding philosophy of this column would be. No specific prediction was made.

In Week 3, I dove into the reasons why yards per carry is almost entirely noise, shared some research to that effect, and predicted that the sample of backs with lots of carries but a poor per-carry average would outrush the sample with fewer carries but more yards per carry.

In Week 4, I talked about yard-to-touchdown ratios and why they were the most powerful regression target in football that absolutely no one talks about, then predicted that touchdowns were going to follow yards going forward (but the yards wouldn't follow back).

In Week 5, we looked at ten years worth of data to see whether early-season results better predicted rest-of-year performance than preseason ADP and we found that, while the exact details fluctuated from year to year, overall they did not. No specific prediction was made.

As I mentioned last week, things look very grim for our yards per carry prediction, but not because we were wrong to bet against yards per carry. Over the last three weeks, our "high ypc" backs average 4.70 yards per carry while our "low ypc" backs average 4.89. Yards per carry is pseudoscience.

No, the real problem here is that our "low-volume" Group A backs went from 13.9 carries per game to 14.1 carries per while our "high-volume" Group B backs fell from 17.7 carries per game all the way down to 12.1 carries per game. The reasons will get a full write-up next week if (as is almost certain) the prediction fails, but there's no single explanation. Instead, of the 10 Group B backs, only two (Darrell Henderson once and Ezekiel Elliott three times) have topped their prior per-game carry average in any game since the prediction. Meanwhile, nine of the eleven Group A backs have combined to top their per-game average seventeen times.

Our second prediction is still trucking along, though. Our "high touchdown" receivers scored 0.40 touchdowns per game while our "low touchdown" receivers scored 0.38 touchdowns per game, but Group B kept their yardage advantage.

The Science of Intuition

One goal of this column is to convince you that regression to the mean is real, it is powerful, and it is everywhere. To explain what it is and how (and why) it works. Another goal is to give you lists of players who are underperforming and players who are overperforming so you can make informed decisions about what to do with them going forward.

But another goal is to equip you with the tools to spot regression in the wild on your own, to help you develop intuitions about what kinds of performances are sustainable and what kinds of performances are unsustainable. Obviously, I'll highlight certain stats and give you my opinions on them. Yards per carry: bad. Yards per touchdown: sustainable, but only within a narrow range from about 100-200. Interception rate: bad. (Sorry, spoiler alert.)

But as years go on, one fact of life in fantasy football is exposure to new statistics. If you listen to football commentary these days you might hear about things like Air Yards, Completion Percentage over Expectation (or CPOE), or Expected Points Added (or EPA). Some of these stats didn't even exist until a few years ago. Are they good? Are they bad?

The gold standard measure of how much a stat might regress is something called stability testing. By comparing performance in one sample to performance in another, we can determine how similar those performances are, how much of a player's performance carries over from one game to the next, from one season to the next. Something like broken tackles, it turns out, is pretty stable. The backs who break a lot of tackles in one year also tend to break a lot of tackles in the next year.

Something like yards per carry, on the other hand, is not stable at all. I've already run down some of the studies, but you can see the results in the predictions from this column, too. Even in this year's prediction, our first-ever yards per carry prediction that is on track to fail, yards per carry itself regressed completely, with our "inefficient" backs averaging more than our "efficient" backs.

But running stability testing is probably going to be beyond the abilities (or the inclinations) of most fantasy football players. (Additionally, just because a statistic is stable doesn't necessarily mean it's useful. Sack rate is one of the most stable quarterback stats, but it's also useless for fantasy football purposes unless you're in the rare league that penalizes quarterbacks for sacks.)

So when you encounter a brand new stat, what can you do to tell if it's a useful stat or not? I'm a big fan of a concept that I call "the leaderboard test", statisticians call "face validity", and the rest of us call "the smell test". Basically, just from looking at a list, how well does it match our intuitions of what that list should look like?

I like a statistic called Adjusted Net Yards per Attempt, or ANY/A. It's a quarterback's yards per attempt, but it gives a 20-point bonus for touchdowns, a 45-point penalty for interceptions, and includes sacks and yards lost to sacks. Why do I like it? Because I think the face validity is really high. Here are the top 10 quarterbacks since the merger in era-adjusted ANY/A (with a 2000-attempt minimum):

1. Steve Young
2. Joe Montana
3. Roger Staubach
4. Peyton Manning
5. Dan Marino
6. Aaron Rodgers
7. Tom Brady
8. Dan Fouts
9. Drew Brees
10. Tony Romo

Maybe that's not a perfect list. Maybe you'd have Tom Brady higher, or Tony Romo lower. But there are seven quarterbacks on the NFL's 100th-anniversary team who played the bulk of their career since the merger, and five of them are on that list, and four of the others (Young, Brees, Rodgers, Fouts) either were or will be first-ballot Hall of Famers. This list has a very high degree of face validity.

Here's the leaderboard for 2020 so far:

1. Matthew Stafford
2. Russell Wilson
3. Dak Prescott
4. Kyler Murray
5. Lamar Jackson
6. Tom Brady
7. Josh Allen
8. Daniel Jones
9. Justin Herbert
10. Jameis Winston

Again, is it perfect? Probably not. But odds are good whatever quarterback you think has been the best in the NFL this season is on that list, and the one potential exception (Pat Mahomes) ranks 11th. The quarterbacks here are typically really good quarterbacks or at least decent quarterbacks who happen to be playing really well.

And because most of the list makes pretty intuitive sense, we should pay extra attention to the surprise entries. Maybe you didn't expect to see Daniel Jones so high, but seeing him sitting between Brady, Allen, and Herbert probably raises your opinion of his play so far this year. Maybe it makes you a bit more impressed with what Jameis Winston has been doing in New Orleans or how well Matt Stafford has the Rams' offense performing.

Let's compare this to another stat. The NFL has been using its player tracking data to create a suite of "Next Gen Stats" to help fans evaluate the game. One stat they created is a measure of the average separation a receiver gets. Here's the Top 10 so far this year:

1. Rondale Moore
2. Freddie Swain
3. Adam Humphries
4. Braxton Berrios
5. Jonnu Smith
6. Maxx Williams
7. Hayden Hurst
8. Quez Watkins
9. Jaylen Waddle
10. Mecole Hardman

Here's the same list from 2020:

1. Deebo Samuel
2. Robert Tonyan Jr
3. Demarcus Robinson
4. David Moore
5. Dawson Knox
6. Dan Arnold
7. George Kittle
8. Drew Sample
9. Allen Lazard
10. Jordan Akins

You would think that a stat that showed how open players were getting would be a good stat, right? But do these lists have face validity? Do they pass the smell test?

Not really. There are some really good players here. There are some really bad players here. There are some one-dimensional deep threats, but there are also short-area separators and yards-after-the-catch specialists. I can probably invent a story to tie all of these guys together. Maybe the good players are here because they're really good at getting wide open. And maybe the bad players are here because the quarterback only looks their way when they're wide open. Maybe.

If you see that a quarterback is having a great season as measured by ANY/A, that should serve as compelling evidence to you that the quarterback is playing really well and you should be predisposed to believe that he'll be able to sustain his production to some extent or another. If you see a receiver is having a great season as measured by average separation, that... shouldn't really move the needle for you at all. That's not really evidence that the receiver is any good or that his level of play is in any way sustainable.

Let's compare this to another stat from Next Gen Stats' list: TAY%, or percent of total air yards. TAY measures how many yards down the field each receiver was on each target (basically how many "air yards" the pass was thrown), totals them up, and then tracks which receivers are getting the highest percentage of their team's total. Here are the current leaders:

1. Brandin Cooks
2. Ja'Marr Chase
3. Davante Adams
4. Terry McLaurin
5. Courtland Sutton
6. DeVonta Smith
7. Justin Jefferson
8. Tyreek Hill
9. Tyler Lockett
10. Mike Williams

That's a much better list! A lot of the players are guys we know are great, and for the surprise entries (Courtland Sutton, DeVonta Smith) just seeing them in this company probably causes us to reevaluate how well they've been playing this season. (Or maybe it causes us to reflect on how bad their competition for targets has been.) TAY% and average separation might both be "advanced stats", but one of them is obviously better than the other at identifying which receivers are actually playing well this year. We can tell just by looking at the respective leaderboards.

Intuitions are fallible, and at the end of the day, they're not as good as rigorous statistical analysis. But rigorous statistical analysis is hard and boring and a lot of work and most of us have better things to do. There's no need to let the perfect be the enemy of the perfectly fine. Raw intuition is an underrated tool for separating the wheat from the chaff and, in a world with an ever-increasing number of new statistics to navigate, quickly settling on what we care about and what's just noise.