Regression Alert: Week 8

Adam Harstad's Regression Alert: Week 8 Adam Harstad Published 10/26/2023

Welcome to Regression Alert, your weekly guide to using regression to predict the future with uncanny accuracy.

For those who are new to the feature, here's the deal: every week, I dive into the topic of regression to the mean. Sometimes, I'll explain what it really is, why you hear so much about it, and how you can harness its power for yourself. Sometimes, I'll give some practical examples of regression at work.

In weeks where I'm giving practical examples, I will select a metric to focus on. I'll rank all players in the league according to that metric and separate the top players into Group A and the bottom players into Group B. I will verify that the players in Group A have outscored the players in Group B to that point in the season. And then I will predict that, by the magic of regression, Group B will outscore Group A going forward.

Crucially, I don't get to pick my samples (other than choosing which metric to focus on). If I'm looking at receivers and Justin Jefferson is one of the top performers in my sample, then Justin Jefferson goes into Group A, and may the fantasy gods show mercy on my predictions.

Most importantly, because predictions mean nothing without accountability, I report on all my results in real time and end each season with a summary. Here's a recap from last year detailing every prediction I made in 2022, along with all results from this column's six-year history (my predictions have gone 36-10, a 78% success rate). And here are similar roundups from 2021, 2020, 2019, 2018, and 2017.


The Scorecard

In Week 2, I broke down what regression to the mean really is, what causes it, how we can benefit from it, and what the guiding philosophy of this column would be. No specific prediction was made.

In Week 3, I dove into the reasons why yards per carry is almost entirely noise, shared some research to that effect, and predicted that the sample of backs with lots of carries but a poor per-carry average would outrush the sample with fewer carries but more yards per carry.

In Week 4, I explained that touchdowns follow yards, but yards don't follow touchdowns, and predicted that high-yardage, low-touchdown receivers were going to start scoring a lot more going forward.

In Week 5, we revisited one of my favorite findings. We know that early-season overperformers and early-season underperformers tend to regress, but every year, I test the data and confirm that preseason ADP is still as predictive as early-season results even through four weeks of the season. I sliced the sample in several new ways to see if we could find some split where early-season performance was more predictive than ADP, but I failed in all instances.

In Week 6, I talked about how when we're confronted with an unfamiliar statistic, checking the leaderboard can be a quick and easy way to guess how prone that statistic will be to regression.

STATISTIC FOR REGRESSION PERFORMANCE BEFORE PREDICTION PERFORMANCE SINCE PREDICTION WEEKS REMAINING
Yards per Carry Group A had 42% more rushing yards per game Group A has 10% more rushing yards per game None (Loss)
Yard-to-TD Ratio Group A had 7% more points per game Group B has 38% more points per game None (Win)
Passing Yards Teams averaged 218.4 yards per game Teams average 221.3 yards per game 10

Our high-touchdown receivers finally came alive in the last week of the prediction, scoring seven touchdowns in ten games, but it was too little, too late. The low-yardage receivers remained low-yardage (going from 57.8 yards per game to 53.0 yards per game), the high-yardage receivers continued getting lots of yards (going from 85.3 to 77.2 yards per game), and while Group A continued doing a better job converting yards into touchdowns (scoring once for every 136 yards compared to once per 168 yards for Group B), the fact that Group B had more yards to convert means they also scored more touchdowns overall (0.46 per game compared to 0.39 per game).

Overall, Group A underperformed their initial average in three out of four weeks since our prediction, while Group B overperformed their starting average in all four weeks.

Our passing yards per game prop had a poor first week as teams averaged 240 yards per game, but a lot of the value proposition here was that passing would decline as the weather turned bad, so there's still a long way to go.


(Most) Quarterback Stats Don't Regress As Much

When explaining how regression works, I mentioned that all production is partly a result of skill (or factors innate to the player) and partly a result of luck (or factors outside the player's direct control). Statistics that are more luck than skill tend to regress quicker and more significantly than statistics that are more skill than luck.

I linked to research by Danny Tuccitto, finding that running backs needed 1,978 carries before their ypc average reached a point where it represented more skill than luck. Using the same methodology, Danny found that a quarterback's yards per attempt average (or YPA) stabilized in just 396 attempts. For a running back, that represents eight years of 250-carry seasons. For a quarterback, that's less than a season's worth of attempts (only one team -- last year's Chicago Bears -- has finished with fewer than 400 pass attempts in the last two seasons).

As a result, you're not going to see me predicting regression very often for quarterback stats like yards per attempt. (In fact, the last time I did so was in 2020 when I predicted-- successfully -- that yards per attempt wouldn't regress.)

But there is one quarterback statistic that I love to badmouth, one that is terrible, horrible, no good, very bad. That statistic is interception rate.

Already a subscriber?

Continue reading this content with a PRO subscription.

Why Does Interception Rate Regress So Much?

If you remember above, statistics that regress more strongly are those that are the result of a relatively higher level of luck than skill. Interception rate does have a skill component. Leaguewide, quarterbacks threw an interception on 2.3% of their attempts last year. For his career, Aaron Rodgers only throws an interception on 1.4% of his attempts, though. Jameis Winston, on the other hand, throws one every 3.4% of attempts. Over 600 pass attempts, that's the difference between 8 interceptions (Rodgers), 14 interceptions (league average), or 20 interceptions (Winston).

We know that's a real difference because the samples involved are so big. Rodgers has thrown more than 7500 career pass attempts. Winston has attempted nearly 3000 passes. The league as a whole attempted more than 18,000 passes last year. These are all significantly greater than the 1,681 attempts that Tuccitto calculated were required for interception rate to stabilize.

But that 1,681 attempt threshold is a lot closer to a running back's 1,978 carry requirement than the 396 attempts necessary for yards per attempt. Why is this?

First: Interceptions Are Heavily Influenced By the Situation

Remember how the league average interception rate last year was 2.4%? On plays where a team was trailing, that rose to 2.5%. When trailing by two scores or more (9 or more points), that rose to 2.6%. When playing with a 2-score lead, that fell to 2.0%. These might not seem like huge differences, but over a 600-attempt season, that's an extra 3 or 4 interceptions.

To some extent, this is selection bias. Consider this 2016 game between the Patriots and the Jets. Quarterbacks threw 0 INTs on 23 attempts with the lead vs. 3 INTs on 24 attempts while trailing. But the Patriots won that game 41-3; all of the attempts with a lead came from Tom Brady, while all of the attempts while trailing belonged to either Bryce Petty or Ryan Fitzpatrick. And it's no surprise that Brady threw fewer interceptions.

To the extent that good quarterbacks spend more time with the lead and good quarterbacks throw fewer interceptions, we should expect quarterbacks with the lead to throw fewer interceptions. But even when you control for the quarterback, the effect persists.

Looking just at Tom Brady, he threw 5771 attempts with the lead in his career and was intercepted on just 1.7% of them. He threw 4373 attempts while trailing and was interception on 2.1% of them. Every other quarterback will show a similar pattern.

And this is good. When a team trails, especially when it trails big or trails late, it needs to take bigger risks to get back into the game. Taking bigger risks will lead to more interceptions, but it will also maximize your chances of a comeback. On the other hand, when a team is ahead, it wants to take fewer risks to make a comeback as difficult as possible for the other team. Some of the best quarterbacks see some of the biggest differences in their interception rate while leading vs. trailing simply because the best quarterbacks tend to be really good at calibrating their risk/reward decision-making to the needs of the moment.

But remember, "luck" is "factors outside of the player's control", and quarterbacks don't have a ton of control over whether their team is leading or trailing at any given point. It depends a lot on how good the opponent is, how well the defense is playing, if special teams are holding up their end of the bargain, etc. So we should expect the situations a quarterback plays in to vary significantly from one sample to the next, and that should impact their expected interception rate.

Second: Interceptions Are Heavily Influenced By the Defender

The quarterback controls whether they throw the ball where a defender can get it or not, but the quarterback has no control over whether the defender actually comes down with that ball.

Brock Purdy threw 163 passes in the first six weeks, and only one of them was intercepted, a rate of 0.6%. (For context, there have only been five seasons in history with an interception rate of 0.6% or lower-- one each by Aaron Rodgers, Tom Brady, Damon Huard, and Nick Foles. How's that for a leaderboard test?)

But Pro Football Focus tracks not just whether a pass was intercepted, but whether it could have been given a reasonable play by the defender, and they charted Purdy with the 4th-most "turnover-worthy throws" of any quarterback in the league. Purdy's interception total wasn't low because he was avoiding bad throws (which would be skill). It was low because defenders were dropping his passes (which was luck). Perhaps unsurprisingly, that luck ran out against the Vikings; Purdy threw two interceptions on 30 attempts, and now his interception rate for the season is 1.6%.

Third: Interceptions Have Tiny Samples

One of the key facts about regression to the mean is the smaller the sample size, the less reliable the data. Remember, the difference between a good quarterback and an average one is often 3 or 4 interceptions over a whole season. That means it's just 1 or 2 interceptions over a half-season. If interceptions occurred on 40% of passes instead of just 2% of passes, they'd undoubtedly stabilize much quicker. (As an illustration: incompletions happen on 40% of pass plays at the college level, and Tuccitto has found that college quarterbacks see their completion percentage stabilize in just 167 attempts, about ten times faster.)

Fourth: We Don't Actually Care About Interceptions as a Rate Stat

Insofar as avoiding interceptions is a skill, that skill manifests as a rate -- interceptions per attempt. That's what I've been talking about to this point. But the reality is most fans don't think of interceptions in those terms; they think about interceptions as a total, instead. If a quarterback throws 2 interceptions on 60 passes, that's not a terrible performance on a per-attempt basis (just a 3.3% interception rate). But most people think of it as a "2-interception game". A quarterback who throws 1 interception on 20 attempts performed much worse (5% interception rate) but generally gets a pass because he only threw one pick.

The number of passes a quarterback attempts varies wildly from week to week based on game situation, injuries to the supporting cast, and defensive matchups. A quarterback who is coming off of a stretch where he averaged 50 attempts per game probably threw a lot of interceptions. He's also likely to throw significantly fewer passes going forward, which will result in fewer interceptions even if the underlying rate stays the same.

So: interception rate is heavily influenced by volume, situation, schedule, and the play of opposing defenders, all of which change drastically from week to week. With so many factors outside of the quarterback's control, it's no wonder interception rates over small samples are so notoriously unreliable.

How unreliable? Let's make a prediction!

First, let's sort all 32 teams from most to fewest interceptions per game. 17 teams are averaging fewer than 0.75 interceptions per game; collectively, they've thrown 64 interceptions at a rate of 0.58 interceptions per game. These teams are the Vikings, Colts, Saints, Chargers, Seahawks, Buccaneers, Cowboys, Panthers, Steelers, Bengals, Lions, Broncos, 49ers, Jaguars, Ravens, Cardinals, and Texans. This is our Group A.

On the other end, 11 teams are averaging at least one interception per game; collectively, they've thrown 85 interceptions at a rate of 1.15 per game. These teams are the Raiders, Browns, Packers, Chiefs, Eagles, Dolphins, Bills, Commanders, Patriots, Bears, and Titans. This is our Group B.

Our Group B is throwing twice as many interceptions per game as Group A. (Really 99.2% more interceptions per game, but what's eight hundredths between friends?) Over the next four weeks, Group A will play 62 games and Group B will play 44 games. I'd bet Group B really does have a marginally higher propensity for interceptions, so I think they'll average more interceptions per game. But I think the difference will be less than 100%. For Group B to throw fewer interceptions than Group A, the per-game difference will need to be less than 40%.

So that's the bet. I predict that teams in Group B will throw fewer interceptions over the next four weeks than teams in Group A, which will require Group B to average less than 40% more interceptions per game going forward.

Photos provided by Imagn Images

More by Adam Harstad

 

Dynasty, in Theory: Do the Playoffs Matter?

Adam Harstad

Should we include playoff performances when evaluating players?

01/18/25 Read More
 

Odds and Ends: Divisional Round

Adam Harstad

Examining past trends to predict the future.

01/17/25 Read More
 

Odds and Ends: Wild Card Weekend

Adam Harstad

Examining the playoff futures and correctly predicting the Super Bowl winner.

01/10/25 Read More
 

Dynasty, in Theory: Evaluating Rookie Receivers

Adam Harstad

Revisiting this year's rookies through the lens of the model

01/09/25 Read More
 

Dynasty, in Theory: Consistency is a Myth

Adam Harstad

Some believe consistency helps you win. (It doesn't.)

01/04/25 Read More
 

Odds and Ends: Week 18

Adam Harstad

How did we do for the year? Surprisingly well!

01/02/25 Read More