The full Stan model is at the bottom, but briefly, we learn a latent playmaking variable for each player that predicts their downstream playmaking stats (e.g. assists, secondary assists, etc.)

Importantly, we saw (unsurprisingly) that increased playmaking is correlated with increased turnovers. Chris Paul was noted as an outlier with high Bayesian playmaking ability, but low empirical turnover rate.

Chris Paul’s Playmaking Through the Years

Here is Chris Paul’s 2013-14 season. Notice how he is near the top in playmaking (far right on the x-axis), but a clear outlier, with a low turnover rate (low on the y-axis)

In the following plot, we’ll overlay all seasons since the 2013-14 season, and it’s remarkable how consistently Chris Paul sits in the bottom right corner. Each season is modeled independently (no parameters are learned across seasons), and each year Chris Paul always stands out with high playmaking, low turnovers.

Here are a few handpicked single-season results:

Uncertainty in playmaking ability

As our models are fully Bayesian, we have uncertainty quantifications in the playmaking latent skill for each season (not shown on the above plots). Showing a few years:

Full Stan Model

data {
    int<lower=0> N;                     // Number of players
    vector[N] possessions;              // Exposure: Possessions
    vector[N] passes_made;              // Exposure: Passes
    
    // Counts (Poisson)
    array[N] int potential_ast;
    array[N] int secondary_ast;
    array[N] int turnovers;
    
    // Continuous (Normal)
    vector[N] ast_pts_created;
    
    // Groups
    int<lower=0> P;
    array[N] int<lower=1, upper=P> position;
}

parameters {
    // Latent Skill (Vision)
    vector[N] skill_raw;             
    vector[P] mu_skill;              
    real<lower=0> sigma_skill;       
    
    // Discrimination & Intercepts (Per 100 Possessions)
    real beta0_pot;     real<lower=0> beta1_pot;
    real beta0_sec;     real<lower=0> beta1_sec;
    
    // Turnovers (Per 100 Passes)
    real beta0_tov;     real<lower=0> beta1_tov;
    
    // Points Created (Continuous)
    real alpha_pts;     real<lower=0> beta_pts;  real<lower=0> sigma_pts;
}

transformed parameters {
    vector[N] skill;
    for (i in 1:N) {
        skill[i] = mu_skill[position[i]] + skill_raw[i] * sigma_skill;
    }
}

model {
    // Hierarchical Skill Priors
    skill_raw ~ std_normal();
    mu_skill ~ normal(0, 1);
    sigma_skill ~ normal(0, 1);
    
    // Priors for Counts (Log scale)
    beta0_pot ~ normal(2.5, 1);    beta1_pot ~ normal(0.5, 0.5);
    beta0_sec ~ normal(0.5, 1);    beta1_sec ~ normal(0.5, 0.5);
    
    // Priors for Turnovers
    beta0_tov ~ normal(-2.5, 1);   beta1_tov ~ normal(0.5, 0.5);
    
    // Priors for Continuous
    alpha_pts ~ normal(30, 10); beta_pts ~ normal(5, 5); sigma_pts ~ normal(5, 5);
    
    // Likelihoods
    // 1. Potential Assists
    potential_ast ~ poisson_log(log(possessions) + beta0_pot + beta1_pot * skill);
    
    // 2. Secondary Assists
    secondary_ast ~ poisson_log(log(possessions) + beta0_sec + beta1_sec * skill);
    
    // 3. Turnovers (Negative relationship with Skill)
    turnovers ~ poisson_log(log(passes_made) + beta0_tov - beta1_tov * skill);
    
    // 4. Assist Points Created
    ast_pts_created ./ (possessions / 100.0 + 1e-9) ~ normal(alpha_pts + beta_pts * skill, sigma_pts);
}

generated quantities {
    vector[N] skill_out = skill;
}

The Model

We start with a hierarchical model pooled by position (full model at the end), where we learn a latent playmaking variable for each player that predicts their downstream playmaking stats including:

1. Potential Assists

2. Secondary Assists

3. Assist Points Created

4. Turnover Rate

Unlike hustle stats which were modeled per minute (for good reason), these are modeled per possession.

The full stan model is at the bottom of this post.

Results

Here’s our top playmakers

But this doesn’t tell the whole story, maybe some of these players are making plays by taking a lot of risk. Comparing it to their turnover rate tells an entirely different story:

This highlights how efficient playmakers like Chris Paul, Tyrese Haliburton, and Tre Jones are. While James Harden, Cade Cunningham, and Trae Young are on the other end of the spectrum.

Chris Paul is an obvious stand-out in the above plot. In the next article, I’m going to highlight how he has (or hasn’t) changed in playmaking ability over time. Spoiler: he hasn’t.

Subscribe now

Full Stan Model

data {
    int<lower=0> N;                     // Number of players
    vector[N] possessions;              // Exposure: Possessions
    vector[N] passes_made;              // Exposure: Passes
    
    // Counts (Poisson)
    array[N] int potential_ast;
    array[N] int secondary_ast;
    array[N] int turnovers;
    
    // Continuous (Normal)
    vector[N] ast_pts_created;
    
    // Groups
    int<lower=0> P;
    array[N] int<lower=1, upper=P> position;
}

parameters {
    // Latent Skill (Vision)
    vector[N] skill_raw;             
    vector[P] mu_skill;              
    real<lower=0> sigma_skill;       
    
    // Discrimination & Intercepts (Per 100 Possessions)
    real beta0_pot;     real<lower=0> beta1_pot;
    real beta0_sec;     real<lower=0> beta1_sec;
    
    // Turnovers (Per 100 Passes)
    real beta0_tov;     real<lower=0> beta1_tov;
    
    // Points Created (Continuous)
    real alpha_pts;     real<lower=0> beta_pts;  real<lower=0> sigma_pts;
}

transformed parameters {
    vector[N] skill;
    for (i in 1:N) {
        skill[i] = mu_skill[position[i]] + skill_raw[i] * sigma_skill;
    }
}

model {
    // Hierarchical Skill Priors
    skill_raw ~ std_normal();
    mu_skill ~ normal(0, 1);
    sigma_skill ~ normal(0, 1);
    
    // Priors for Counts (Log scale)
    beta0_pot ~ normal(2.5, 1);    beta1_pot ~ normal(0.5, 0.5);
    beta0_sec ~ normal(0.5, 1);    beta1_sec ~ normal(0.5, 0.5);
    
    // Priors for Turnovers
    beta0_tov ~ normal(-2.5, 1);   beta1_tov ~ normal(0.5, 0.5);
    
    // Priors for Continuous
    alpha_pts ~ normal(30, 10); beta_pts ~ normal(5, 5); sigma_pts ~ normal(5, 5);
    
    // Likelihoods
    // 1. Potential Assists
    potential_ast ~ poisson_log(log(possessions) + beta0_pot + beta1_pot * skill);
    
    // 2. Secondary Assists
    secondary_ast ~ poisson_log(log(possessions) + beta0_sec + beta1_sec * skill);
    
    // 3. Turnovers (Negative relationship with Skill)
    turnovers ~ poisson_log(log(passes_made) + beta0_tov - beta1_tov * skill);
    
    // 4. Assist Points Created
    ast_pts_created ./ (possessions / 100.0 + 1e-9) ~ normal(alpha_pts + beta_pts * skill, sigma_pts);
}

generated quantities {
    vector[N] skill_out = skill;
}

The Model

As always, the Stan full model is at the bottom of the post. For each player, we learn two hierarchical latent variables that describe their interior and exterior hustle. This is very much in the spirit of our previous latent skill model for NBA shooting, but much less rigorous. These latent variables are simultaneously used to predict various hustle stats using a poisson distribution.

These are the hustle stats we included in our first iteration of the model.

Interior Hustle Stats:

• Screen Assists

• Box Outs

• Contested Shots

Exterior Hustle Stats:

• Deflections

• Charges Drawn

• Loose Balls Recovered

• Defensive Speed

What’s funny, to me at least, is our poisson estimates use per-minute stats, which is usually a pretty naive way to model NBA performance. But in the case of hustling, I like it. What you want is someone who’s grinding every minute of the game.

Top Hustlers

Using this, we can get the bayesian posterior estimates for the top interior and exterior hustlers:

And we can break each player out on both dimensions. In line with expectations, centers have higher interior hustle latent skills, and guards have higher exterior hustle latent skills. The players in the top right are those who score high in both regards.

It’s pretty obvoius Paul Reed wins this model.

Let me know if there are any players you are particularly interested in, and I’ll highlight their posterior estimates.

Full Stan Model

/* 
====================================================================
   MODEL 1: PERIMETER HUSTLE (Guards/Wings)
   Features: Deflections, Loose Balls, Charges (Poisson) + Speed (Normal)
====================================================================
*/

data {
    int<lower=0> N;                     // Number of players
    vector[N] minutes;                  // Exposure
    
    // Counts (Poisson)
    array[N] int deflections;
    array[N] int loose_balls;
    array[N] int charges;
    
    // Continuous (Normal)
    vector[N] speed;       // Avg Speed Def
    vector[N] distance;    // Dist Miles Def
    
    // Groups
    int<lower=0> P;
    array[N] int<lower=1, upper=P> position;
}

parameters {
    // Latent Skill
    vector[N] skill_raw;             
    vector[P] mu_skill;              
    real<lower=0> sigma_skill;       
    
    // Discrimination & Intercepts
    real beta0_deflect; real<lower=0> beta1_deflect;
    real beta0_loose;   real<lower=0> beta1_loose;
    real beta0_charge;  real<lower=0> beta1_charge;
    
    // Continuous Params (Speed ~ Normal(alpha + beta*skill, sigma))
    real alpha_speed; real<lower=0> beta_speed; real<lower=0> sigma_speed;
    real alpha_dist;  real<lower=0> beta_dist;  real<lower=0> sigma_dist;
}

transformed parameters {
    vector[N] skill;
    for (i in 1:N) {
        skill[i] = mu_skill[position[i]] + skill_raw[i] * sigma_skill;
    }
}

model {
    // Priors
    skill_raw ~ std_normal();
    mu_skill ~ normal(0, 1);
    sigma_skill ~ normal(0, 1);
    
    // Priors for Counts
    beta0_deflect ~ normal(-3, 2); beta1_deflect ~ normal(0.5, 0.5);
    beta0_loose ~ normal(-4, 2);   beta1_loose ~ normal(0.5, 0.5);
    beta0_charge ~ normal(-5, 2);  beta1_charge ~ normal(0.5, 0.5);
    
    // Priors for Continuous
    // Speed is around 3.5-4.5 mph
    alpha_speed ~ normal(4, 1); beta_speed ~ normal(0.1, 0.1); sigma_speed ~ normal(0.5, 0.5);
    
    // Likelihoods
    deflections ~ poisson_log(log(minutes) + beta0_deflect + beta1_deflect * skill);
    loose_balls ~ poisson_log(log(minutes) + beta0_loose + beta1_loose * skill);
    charges ~ poisson_log(log(minutes) + beta0_charge + beta1_charge * skill);
    
    // Continuous Likelihood
    speed ~ normal(alpha_speed + beta_speed * skill, sigma_speed);
}

generated quantities {
    vector[N] skill_out = skill;
}

/* 
====================================================================
   MODEL 2: INTERIOR HUSTLE (Bigs)
   Features: Box Outs, Contests, Screen Assists (Poisson)
====================================================================
*/

data {
    int<lower=0> N;                     // Number of players
    vector[N] minutes;                  // Exposure
    
    // Counts (Poisson)
    array[N] int box_outs;
    array[N] int contest_2pt;
    array[N] int contest_3pt;
    array[N] int screen_assists;
    
    // Groups
    int<lower=0> P;
    array[N] int<lower=1, upper=P> position;
}

parameters {
    // Latent Skill
    vector[N] skill_raw;             
    vector[P] mu_skill;              
    real<lower=0> sigma_skill;       
    
    // Discrimination & Intercepts
    real beta0_box;     real<lower=0> beta1_box;
    real beta0_cont2;   real<lower=0> beta1_cont2;
    real beta0_cont3;   real<lower=0> beta1_cont3;
    real beta0_screen;  real<lower=0> beta1_screen;
}

transformed parameters {
    vector[N] skill;
    for (i in 1:N) {
        skill[i] = mu_skill[position[i]] + skill_raw[i] * sigma_skill;
    }
}

model {
    // Priors
    skill_raw ~ std_normal();
    mu_skill ~ normal(0, 1);
    sigma_skill ~ normal(0, 1);
    
    // Priors for Counts
    beta0_box ~ normal(-3, 2);    beta1_box ~ normal(0.5, 0.5);
    beta0_cont2 ~ normal(-2, 2);  beta1_cont2 ~ normal(0.5, 0.5);
    beta0_cont3 ~ normal(-2, 2);  beta1_cont3 ~ normal(0.5, 0.5);
    beta0_screen ~ normal(-3, 2); beta1_screen ~ normal(0.5, 0.5);
    
    // Likelihoods
    box_outs ~ poisson_log(log(minutes) + beta0_box + beta1_box * skill);
    contest_2pt ~ poisson_log(log(minutes) + beta0_cont2 + beta1_cont2 * skill);
    contest_3pt ~ poisson_log(log(minutes) + beta0_cont3 + beta1_cont3 * skill);
    screen_assists ~ poisson_log(log(minutes) + beta0_screen + beta1_screen * skill);
}

generated quantities {
    vector[N] skill_out = skill;
}

Here, we develop a more unified model that hierarchically learns each player’s latent shooting ability, by incorporating their FT shooting and 3PT shooting data from the 2025-26 season so far.

The Model

The full model is at the bottom of this post, but at a high level we’re using a hierarchical binomial model pooled by position to learn a latent shooting ability variable for each player. The shooting ability is used downstream to predict both their 3PT and FT shooting.

Here, on the x-axis we’re plotting the learned latent shooting skill, vs the actual/modeled FT and 3PT% of each player. To be honest, I was initially surprised at how well this works for predicting free throw shooting. 3PT shooting is a bit disappointing.

Top shooting talent in the NBA

Here’s our estimate for latent shooting skill for the top 20 players in the NBA, and then broken out by position below.

There’s a lot more we want to do. There are so many more events that our latent shooting ability variable can be used to predict (corner 3s, mid range shots, etc), which can help tighten up our estimates. For example, knowing how well someone shoots from mid range should help tighten up our estimate for how well they shoot from the free throw line. I’m looking forward to incorporating this data. More broadly, I’m becoming interested in these latent variables/vectors that can be used to predict specific events.

Full Stan Model

// Unified Shooting Ability Model
// Estimates a single latent "skill" per player that drives both FT% and 3P%

data {
  int<lower=0> N;                 // number of players
  int<lower=1> P;                 // number of positions (3)
  array[N] int<lower=1,upper=P> position;
  
  // FT Data
  array[N] int<lower=0> fta;
  array[N] int<lower=0> ftm;
  
  // 3PT Data
  array[N] int<lower=0> fg3a;
  array[N] int<lower=0> fg3m;
}

parameters {
  // Latent Skill (hierarchical by position)
  // We model this as a Z-score (standard normal) for identifiability
  vector[P] mu_skill;             // Average skill per position
  vector<lower=0>[P] sigma_skill; // Variance in skill per position
  vector[N] raw_skill;            // Player-specific skill deviation
  
  // Shot Mechanics (Intercepts & Slopes)
  real beta0_ft;                  // Base difficulty FT
  real<lower=0> beta1_ft;         // How much skill helps FT (discrimination)
  
  real beta0_3p;                  // Base difficulty 3PT
  real<lower=0> beta1_3p;         // How much skill helps 3PT (discrimination)
}

transformed parameters {
  vector[N] skill;
  // Non-centered parameterization for skill
  for (n in 1:N) {
    skill[n] = mu_skill[position[n]] + sigma_skill[position[n]] * raw_skill[n];
  }
}

model {
  // 1. Priors
  // Skill priors
  mu_skill ~ normal(0, 1);
  sigma_skill ~ normal(1, 0.5);
  raw_skill ~ std_normal();
  
  // Shot parameter priors
  beta0_ft ~ normal(1.1, 0.5);    // ~75% baseline
  beta1_ft ~ normal(1, 0.5);      // Positive correlation
  
  beta0_3p ~ normal(-0.5, 0.5);   // ~38% baseline
  beta1_3p ~ normal(1, 0.5);      // Positive correlation

  // 2. Likelihoods
  ftm ~ binomial_logit(fta, beta0_ft + beta1_ft * skill);
  fg3m ~ binomial_logit(fg3a, beta0_3p + beta1_3p * skill);
}

generated quantities {
  vector[N] ft_pct;
  vector[N] fg3_pct;
  
  for (n in 1:N) {
    ft_pct[n] = inv_logit(beta0_ft + beta1_ft * skill[n]);
    fg3_pct[n] = inv_logit(beta0_3p + beta1_3p * skill[n]);
  }
}

Here, we adapt this same model to 3PT shooting.

Model

The full Stan model is at the bottom of this post. At a high level, we’re using a hierarchical binomial model, pooled by player position (guard, forward, center), using the following features:

• Height

• Experience

• Draft Position

• Weight

• International vs US

Remember, as this is hierarchical, when players have few attempts, the model basically shrugs and assumes they are going to look like other players with their features.

Results

Player Distribution

Here’s the distribution of 3PT% estimates for all players in the league, broken out by position. This distribution is so interesting to me. Tons of variance, which makes sense, but there’s this long-tail shoulder on the guards above 38%.

Now, lets inspect each feature.

Height: No definitive effect, Taller centers might be affected

Experience: Obvious effect

Probably survivorship bias, but interesting nonetheless.

Draft position: Surprising, but makes sense

Later draft positions, especially for big men, predicts higher three point shooting. If you think about the archetype of centers that are drafted early vs later (and who is picked up undrafted), this isn’t as counter-intuitive as it seems. To me, the main question is, why didn’t we see this in our FT modeling.

Weight: Nothing to write home about

A lot of density for the Center position’s weight goes negative, which I can buy. Guards, perfectly centered on zero, which also makes sense. Lighter centers might be better at shooting threes, while weight doesn’t affect guards as much.

International vs US

Nothing to see here.

Full Stan Model

// Extended hierarchical binomial model for 3-point shooting
// Position-specific coefficients for 5 regressors:
// height, experience, draft position, weight, international

data {
  int<lower=0> N;                 // number of players
  int<lower=1> P;                 // number of positions (3)
  array[N] int<lower=1,upper=P> position;
  array[N] int<lower=0> fg3a;     // CHANGED: fta -> fg3a
  array[N] int<lower=0> fg3m;     // CHANGED: ftm -> fg3m
  
  // Regressors (all standardized)
  vector[N] height_z;
  vector[N] exp_z;
  vector[N] draft_z;
  vector[N] weight_z;
  vector[N] is_intl;              // binary: 0=USA, 1=international
}

parameters {
  vector[P] mu;                   // position intercepts (logit scale)
  
  // Position-specific coefficients
  vector[P] beta_height;
  vector[P] beta_exp;
  vector[P] beta_draft;
  vector[P] beta_weight;
  vector[P] beta_intl;
  
  vector<lower=0>[P] sigma;       // position residual sds
  vector[N] theta;                // player abilities (logit scale)
}

model {
  // Priors on intercepts
  // CHANGED: Centered at -0.5 (approx 38% 3PT) instead of 1.1 (75% FT)
  mu ~ normal(-0.5, 0.5);          
  
  // Priors on coefficients (weakly informative)
  beta_height ~ normal(0, 0.3);
  beta_exp ~ normal(0, 0.3);
  beta_draft ~ normal(0, 0.3);
  beta_weight ~ normal(0, 0.3);
  beta_intl ~ normal(0, 0.3);
  
  sigma ~ exponential(2);
  
  // Player abilities with all regressors
  for (n in 1:N) {
    real linear_pred = mu[position[n]] +
                       beta_height[position[n]] * height_z[n] +
                       beta_exp[position[n]] * exp_z[n] +
                       beta_draft[position[n]] * draft_z[n] +
                       beta_weight[position[n]] * weight_z[n] +
                       beta_intl[position[n]] * is_intl[n];
    theta[n] ~ normal(linear_pred, sigma[position[n]]);
  }
  
  // Likelihood
  fg3m ~ binomial_logit(fg3a, theta);
}

generated quantities {
  vector[N] fg3_pct;
  vector[P] mu_pct;
  
  fg3_pct = inv_logit(theta);
  mu_pct = inv_logit(mu);
}

Game Predictions: Full Bayesian Posterior Distributions

Read more

Let’s double click into that.1

Note, this has been a draft in my newsletter since 2022. Finally publishing it.

Simulations

We ran some very simple simulations. The simulations over-simplify the game of basketball, but that’s the point. The people who make claims about three point shots increasing variance in basketball would accept the terms of these simulations.

Here’s the details of our simulations:

• Each team gets to make 100 shots a game

• Each team can decide how they want to distribute the shots (number of 3PT shots, number of 2PT shots)

• 2PT shots and 3PT shots have the same expected value

• Number of made shots (and therefore, final score) is drawn from a binomial distribution

That’s it. Like we said, this is a very unrealistic view of basketball. But, the variance-believers essentially use those assumptions when they make claims like this:

While a triple-heavy attack is, in game theory parlance, a dominant strategy, it’s also a high-variance one. 3s are worth more than other shots and they go in less frequently, reducing the predictability of individual games. - hoop76.com from 7 years ago.

Here, we start by comparing two teams: 1 team takes all 2PT shots, and the other team takes all 3PT shots. Since 2PT shots and 3PT shots have the same expected value, both teams are scoring about 100 points on average. But look at the variance. The 3 point shooting teams have a higher probability of going off for >120 points in a game. This is exactly in line with what the variance-believers are talking about!

But teams aren’t shooting all 3’s or all 2’s. The problem arises when we start looking at more realistic looking numbers. Here we’re comparing a team that takes 20% of their shots from 3 to a team that takes 40% of their shots from 3. You can already see the myth falling apart. The team that takes double the amount of shots from 3 hardly has a heavier tail.

But let’s look at it from a strategic point of view. If you are a worse team, should you take more 3PT shots to increase your variance and increase your chance of beating a better team?

Here’s two teams that take the same number of threes, but one team is better than the other team by 5 points on average.

In the plot above, you can see that the worse team can beat the better team due to chance, even without increasing their variance by shooting more threes. But what if the worse team tries to increase their variance by taking more threes? For the worse team, we’ve simulated games where they take between 20% of their shot from 3 and 60% of their shots from 3. And then for each volume of 3PT shoots, we look at the probability they beat the better team

As expected, taking more 3s increases the chance that the worse team can beat the better team! But look closely at the y-axis. Increasing your 3PT shooting from 20% of shots to 60% of shots only increases your probability of winning from 36.7% to 37.5%. Again, the point is that once you use realistic numbers, the effect size is hardly noticeable.

So what’s really going on here?

Where did this myth come from? I think there’s at least two plausible explanations:

1. There’s a ton of variance in basketball. When people get exposed to the variance, they naturally want to attribute it to things. And since statistically its true that 3PT shooting increases variance (even if it hardly explains the variance they’re seeing), it’s an easy target.

2. 3PT shooting increases variance in ways not captured by our simulations. But the point is that 3PT shooting doesn’t materially increase variance simply due to it being a lower probability shot worth more points.

Our money is on the second one.

Looking ahead

I’ve been meaning to iterate on our stat prediction model by using a negative-binomial model to have more control over the variance. I’m looking forward to those results.

We’ve fired back up our Bayesian hierarchical models. Here is our posterior estimates for tonight’s games:

And a compact summary:

Background Reading

Many of our previous posts have detailed how we predict NBA game outcomes. In the future, we’ll go into more details into our updated model. We won’t rehash it here, but here’s some links:

Then we incrementally added in their position, since it makes more sense to assume guards shoot like guards if you don’t know anything else about them.

Here, we take it one step further. Actually 5 steps further, just dumping in the next easiest features into the model.

This was partially inspired by our friend who has a solid NBA analysis newsletter

The Model

Incrementally improving on our free throw modeling, we added the following position-specific features:

• Height

• Experience

• Draft Position

• Weight

• International vs US

The Players

Here’s the posterior broken out by position:

Height: Smaller is Better, But Tons of Uncertainty

The estimate for the effect of height is negative, but the uncertainty is huge. Notably, it’s reasonably certain for guards.

Broken out by position, plotting our estimate of the effect of height on top of the raw data:

Experience: More → Better

This isn’t to say more experience makes you better. But if all you know is a player’s position and number of years in the NBA, you know more than nothing.

Draft Position: Another Obvious Effect

Probably don’t need much commentary on this one.

Weight: Heavy Forwards Struggle

I’m curious what your thoughts on weight are. I’ll hold off on my interpretation, but if you leave a comment, I promise to engage.

International vs US

It doesn’t matter.

Top Shooters

Look how much we’ve tightened up our estimates.

Bottom Shooters

Looking Ahead

I’m so thankful for everyone that DM’d me over the last few weeks.

I probably won’t be posting any more free throw modeling for a bit. I have some really off-the-wall ideas that I’ve been modeling, and I want to share that next, I think.

The full model is at the bottom of the post, but to explain it simply: for each position we learn the distribution of free throw shooting ability, which helps inform estimates for each player of that position.

If we see someone shoot 13 out of 13 so far this season, we expect their true FT% to be different if that player is a center vs a guard. This model accounts for that.

Position Distributions

This plot shows each player’s learned posterior FT%, colored by position.

Nothing here is surprising, but it’s nice confirmation of what we expect. Importantly, the variance on centers is larger than other positions. To me, not unexpected, but the magnitude of the phenomenon is more than I would have guessed.

Best Shooters by Position

Some things to note:

1. It’s probably not worth it to walk through every player, but take some time to look at everyone.

2. I like noting when the Bayesian estimates are ranked differently than the empirical estimates. E.g. Cam Spencer. It always corresponds to a large uncertainty in the player’s Bayesian estimate.

3. Al Horford.

Worst Shooters by Position

1. The uncertainties here are larger.

2. Since these are the extremes, they are almost always pulled towards their positional means.

3. Rudy Gobert.

Looking Ahead

I’ll probably continue to make this incrementally more complicated until I get interested in a new topic. Any ideas to make it incrementally complicated are welcome! Let me know.

Model

// Hierarchical binomial model with position-level priors
// Each position (Guard, Forward, Center) has independent mu and sigma

data {
  int<lower=0> N;                 // number of players
  int<lower=1> P;                 // number of positions (3)
  array[N] int<lower=1,upper=P> position;  // position for each player
  array[N] int<lower=0> fta;      // free throw attempts
  array[N] int<lower=0> ftm;      // free throws made
}

parameters {
  vector[P] mu;                   // position-level means (logit scale)
  vector<lower=0>[P] sigma;       // position-level sds
  vector[N] theta;                // player-specific ability (logit scale)
}

model {
  // Position-level priors (independent for each position)
  mu ~ normal(1.1, 0.5);          // ~75% FT prior on probability scale
  sigma ~ exponential(2);

  // Players drawn from their position’s distribution
  for (n in 1:N) {
    theta[n] ~ normal(mu[position[n]], sigma[position[n]]);
  }

  // Likelihood
  ftm ~ binomial_logit(fta, theta);
}

generated quantities {
  vector[N] ft_pct;
  vector[P] mu_pct;               // position means on probability scale
  ft_pct = inv_logit(theta);
  mu_pct = inv_logit(mu);
}

As usual, this is just a starting point, and in follow up posts, we’ll add more complexity as we go.

The Model

We’ll use a simple binomial hierarchical model (full model below). Where each player’s free throw ability is estimated from a league-wide distribution where the mean and variance of the league-wide distribution are learned from the data.

Importantly, if a player has only taken a handful of shots this season, the model assumes that they are basically just an average player. So naturally, we don’t need to do those arbitrary things like only limiting it to players with at least 50 free throw attempts.

Bayesian free throw estimates vs empirical estimates

Here’s a plot of our bayesian estimates for each player in the league (blue) vs their corresponding empirical estimates (orange).

All those 100% shooters? All the extremes? The model is just saying, there’s no way they’re that good/bad.

Best Free Throw Shooters

Here are the top 10 shooters this season. The blue dot shows the posterior median, compared to their empirical free throw percentage (orange diamond). I think there’s three important things to point out:

1. There’s a lot of uncertainty. To answer the question in the title of this post: I don’t know.

2. The model may be too conservative. Future posts will follow up.

3. Walter Clayton Jr. is empirically very good, but this clearly shows we just haven’t seen enough to know for sure, yet.

Worst Free Throw Shooters

Two notes:

1. The model’s like: No, Mitchell Robinson can’t be that bad.

2. There is so much certainty in how bad Rudy Gobert is.

Looking Ahead

I feel like I’m coming out of a coma. I wanted to start with something simple just to get pen on to paper. In the next post we’ll see what we can do about Kawhi Leonard.

Model

data {
  int<lower=0> N;                 // number of players
  array[N] int<lower=0> fta;      // free throw attempts
  array[N] int<lower=0> ftm;      // free throws made
}

parameters {
  real mu;                        // population mean (logit scale)
  real<lower=0> sigma;            // population sd
  vector[N] theta;                // player-specific ability (logit scale)
}

model {
  // Priors
  mu ~ normal(1.1, 0.5);          // ~75% FT prior on probability scale
  sigma ~ exponential(2);

  // Hierarchical prior for player abilities
  theta ~ normal(mu, sigma);

  // Likelihood
  ftm ~ binomial_logit(fta, theta);

}

generated quantities {
  vector[N] ft_pct;
  ft_pct = inv_logit(theta);      // transform to probability scale
}

Or, more concretely: How likely are the Knicks to win against the 76ers when they are up by 5 at halftime?

Specifically, we built a fully Bayesian hierarchical model that incorporates team offensive strength, team defensive strength, and the halftime score to predict the final winner in an NBA game.

As always, the full model is at the bottom of the post, but put simply: this is an extension of our previous game prediction model. It adds a second component to the likelihood that uses the halftime score to predict the final winner.

Win probabilities for equal-strength teams

What if two perfectly matched strength teams are playing? Since we have a fully Bayesian model, we can look not only at point estimates but the entire distribution of probabilities learned by the model.

Seen above, the home team has between a 0.2 and 0.8 probability of winning (when playing an equal strength opponent), depending on what the halftime score is. Personally, one point at halftime had a larger effect than I was expecting.

Team-vs-team match-ups at different halftime scores.

We can also look at the probability that each home team is to win against their opponent for any halftime score. First, starting with a tied halftime score:

But we can also adjust the halftime score and see how the win probabilities change. Here’s what happens when the home team is winning by 5 at halftime:

Any home team, any away team, any halftime score

We can query the model for any combination of teams and halftime score. We’ll get the point estimate but also the measure of our uncertainty in the prediction.

Going back to our original example: How likely are the Knicks to win against the 76ers when they are up by 5 at halftime? Here, we’re plotting their win probability distribution when they are tied at halftime and when they are winning by 5 at halftime:

The model

// Hierarchical IRT regression
//
// This models the points of home and away teams
// as a function of the latent offensive and defensive 
// strength of the teams.
//
// Specifically, this version tries to predict the game winner
// given the first half score.

data {
    // Number of games
    int<lower=1> N_games;

    // Number of teams in the league
    int<lower=1> N_teams;

    // Home and away points scored in each half
    array[N_games] int<lower=0> home_points_h1;
    array[N_games] int<lower=0> away_points_h1;
    array[N_games] int<lower=0> home_points;
    array[N_games] int<lower=0> away_points;    

    // Team index for each game
    array[N_games] int<lower=1, upper=N_teams> home_team;
    array[N_games] int<lower=1, upper=N_teams> away_team;

    // Indicator variable for home team winning
    array[N_games] int<lower=0, upper=1> home_win;

    // First Half Difference in points
    array[N_games] int first_half_diff;
}

transformed data {
    // Section 3.3.1 https://betanalpha.github.io/assets/case_studies/prior_modeling.html
    // Threshold for home field advantage
    // 99% of the density should be between 0 and this value    
    real home_field_advantage_threshold = 5;
    real home_field_advantage_prior_sigma = home_field_advantage_threshold / 2.57;
}

parameters {
    // Latent offensive and defensive strength of each team
    // Hierarchical prior
    vector[N_teams] theta_offense;
    vector[N_teams] theta_defense;
    real theta_offense_bar;
    real theta_defense_bar;
    real<lower=0> sigma_offense_bar;
    real<lower=0> sigma_defense_bar;

    // Noise in the points (same for home and away teams)
    real<lower=0> sigma_points;

    // Home field advantage is extremely unlikely to be negative
    real <lower=0> home_field_advantage;

    // Home field advantage of home team winning
    real home_field_advantage_classification;

    // Impact of first half score on final score
    real theta_first_half;

    // Impact of team strength on home team winning
    real theta_team_strengths;
}

model {

    // Prior Modelinng

    // Average strength of the teams
    theta_offense_bar ~ normal(110, 6.67);

    // Home field advantage
    // Put 99% of dennsity between 0 and input {home_field_advantage_threshold}
    home_field_advantage ~ normal(0, home_field_advantage_prior_sigma);
    home_field_advantage_classification ~ normal(0, home_field_advantage_prior_sigma);

    // Variations of the teams strength
    sigma_offense_bar ~ normal(0, 12);
    sigma_defense_bar ~ normal(0, 12);

    // Individual team strength
    theta_offense ~ normal(theta_offense_bar, sigma_offense_bar);
    theta_defense ~ normal(0, sigma_defense_bar);

    // Gaussian noise in the points
    sigma_points ~ normal(0, 7);

    // Imact of first half score on home team winning
    theta_first_half ~ normal(0, 3);

    // Impact of team strength on home team winning
    theta_team_strengths ~ normal(0, 10);

    // Likelihood
    for(game in 1:N_games) {
        // Team points modeled as gaussian
        real home_points_regression = home_field_advantage + 
                                      theta_offense[home_team[game]] + 
                                      theta_defense[away_team[game]];
        real away_points_regression = theta_offense[away_team[game]] + 
                                      theta_defense[home_team[game]];
        home_points[game] ~ normal(home_points_regression, sigma_points);
        away_points[game] ~ normal(away_points_regression, sigma_points);   

        // Probability home team wins
        real home_win_regression = home_field_advantage_classification + 
                                   theta_team_strengths * (theta_offense[home_team[game]] + 
                                                           theta_defense[away_team[game]] - 
                                                           theta_offense[away_team[game]] - 
                                                           theta_defense[home_team[game]]) +
                                   theta_first_half * first_half_diff[game];
        home_win[game] ~ bernoulli_logit(home_win_regression);

    }
}

generated quantities {

    // Remove the mean from the latent variables
    vector[N_teams] theta_defense_centered;

    for (i in 1:N_teams) {
        theta_defense_centered[i] = theta_defense[i] - mean(theta_defense);
    }

    vector[N_teams] theta_offense_centered;

    for (i in 1:N_teams) {
        theta_offense_centered[i] = theta_offense[i] - mean(theta_offense);
    }
}

Usually our posts are self-contained, but unfortunately, this one won’t make much sense without first catching up on the previous post.

Fixing priors by inspecting their downstream effects

We left off by fixing sigma_offense_bar from having unreasonably fat tails. This is a parameter that says how much variance in offensive strength there is across different teams.

But now the question is: is the refined prior reasonable? It tapers off around 20. But what does 20 even mean? Should it taper off around 100?

The parameter is too abstract to have any intuition about it directly, but we can inspect the downstream consequences of its distribution. Specifically, we can look at the resulting distribution of team offensive strengths (in this case, points per game) we see using this prior distribution.

So basically, teams are constrained between averaging 85 points per game and 135 points per game. That’s actually not too bad, but it’s a bit too restrictive, right? Let’s not rule out that a team might be truly awful. It’s possible, right?

If we give sigma_offense_bar a bit more breathing room, we end up with this refined distribution for team offensive strength:

Pretty similar, but I’m a bit more comfortable having some extreme values be possible.

As a reminder, we aren’t trying to perfectly predict team offensive strengths. We’re trying to define reasonable bounds and have 99% of our prior distribution in reasonable values.

Pushing Priors all the way down

We can keep running through this pushdown strategy for all parameter prior distributions. For example: sigma_points is a parameter that controls how much variance there is in points scored for a team given everything we know about the team’s offense and the opposing team’s defense.

I couldn’t tell you what a reasonable prior model on that parameter is, but what I can tell you is if the resulting implication of that prior makes any sense. For example, we can simulate a season of games under a prior model and look at the resulting distribution of points scored per team per game:

Maybe that could use a bit more breathing room? I think it’s pretty good.

As a quick recap:

• For straightforward priors that you have direct domain knowledge on: put an uniformative prior with 99% of density within bounds of what you deem possible.

• For abstract priors: evaluate the downstream consequences. Again, put 99% of the density of the downstream consequences within what you deem possible.

Looking ahead

I wanted to keep working on this model iteratively. But too many people have told me it’s going too slowly. So I think I’ll mix it up and show some other models while working through the same types of iterations.

Stan Model

// Heirarchical IRT regression
//
// This models the points of home and away teams
// as a function of the latent offensive and defensive 
// strength of the teams.
//
// Specifically, this version tries to improve on the
// prior modeling.

data {
    // Number of games
    int<lower=1> N_games;

    // Number of teams in the league
    int<lower=1> N_teams;

    // Home and away points scored in each game
    array[N_games] int<lower=0> home_points;
    array[N_games] int<lower=0> away_points;

    // Team index for each game
    array[N_games] int<lower=1, upper=N_teams> home_team;
    array[N_games] int<lower=1, upper=N_teams> away_team;

    // Threshold for home field advantage
    // 99% of the density should be between 0 and this value
    real home_field_advantage_threshold;
}

transformed data {
    // Section 3.3.1 https://betanalpha.github.io/assets/case_studies/prior_modeling.html
    real home_field_advantage_prior_sigma = home_field_advantage_threshold / 2.57;
}


generated quantities {
   
    // Prior Modeling

    // Average strength of the teams
    real theta_offense_bar;
    theta_offense_bar = normal_rng(110, 6.67);

    // Home field advantage
    // Put 99% of dennsity between 0 and input {home_field_advantage_threshold}
    real <lower=0> home_field_advantage;
    home_field_advantage = abs(normal_rng(0, home_field_advantage_prior_sigma));

    // Variations of the teams strength
    real<lower=0> sigma_offense_bar;
    real<lower=0> sigma_defense_bar;   
    sigma_offense_bar = abs(normal_rng(0, 12));
    sigma_defense_bar = abs(normal_rng(0, 12));

    // Individual team strength
    real theta_offense;
    real theta_defense;    
    theta_offense = normal_rng(theta_offense_bar, sigma_offense_bar);
    theta_defense = normal_rng(0, sigma_defense_bar);

    // Gaussian noise in the points
    real<lower=0> sigma_points;
    sigma_points = abs(normal_rng(0, 7));

    real home_points_regression = home_field_advantage + theta_offense + theta_defense;
    real away_points_regression = theta_offense + theta_defense;
    real home_points_sample = normal_rng(home_points_regression, sigma_points);
    real away_points_sample = normal_rng(away_points_regression, sigma_points); 

}

In our last post, we looked in depth at a single prior and how to align it with our expectations. Specifically, we looked at home field advantage where developing a prior model was fairly formulaic: decide which values would be ridiculous (in this case, negative home field advantage or greater than 5 points home field advantage) and then set our prior to have 99% of density between these bounds.

In this post, we’ll apply this formula to some more priors and see when the method breaks down. In the following post, we’ll overcome the limitations of this simple formula.

At the bottom of this post, our Stan model was rewritten to get the entire distribution for every prior. Looking at those distributions, we see our old friend home field advantage. You can see the bounds we introduced last time: A hard threshold at 0 points, and 99% of the density falling below 5 points.

Prior modeling offensive strength

Let’s look at another prior: average team offensive strength. This parameter essentially describes the points per game of an average NBA team.

Here’s the prior we used:

theta_offense_bar ~ normal(116, 10);

We used our old lazy trick: set the mean to something reasonable and set the variance large to give our laziness some breathing room.

But this prior ends up being pretty unreasonable. Do you really expect an average team to score 95 points a game? What about 135? Let’s tighten up our prior to match our expectations. Following the same formula from last time: what values would be completely unreasonable? Once we define those, put 99% of our prior density between those values.

So what values are ridiculous for an average team’s points per game? In the last twenty years of data, it would look extremely unlikely for an average team to score less than 90 points or more than 130 points:

A prior with 99% density between 90 and 130 corresponds to:

theta_offense_bar ~ normal(110, 6.67);

Comparing this to our old priors looks much better:

Prior modeling the variance in offensive strength

There’s another parameter in our model that describes how much variation in offensive strength there is across teams. If this parameter is 0, all teams have the same offensive strength. If this value is really high, teams have wildly different offensive strengths. Here’s the prior model we used:

real<lower=0> sigma_offense_bar;  
sigma_offense_bar ~ cauchy(0, 5);

A half-cauchy centered on 0. Why cauchy? Cauchy is far heavier-tailed than the normal distribution. When we set 99% of the density to align with our expectations, the other 1% is very flat and extends well beyond our reasonable thresholds. Effectively, it says: “if we’re wrong about our thresholds, go nuts. The true might be anything”. As opposed to the more moderately-tailed normal distribution that says: “if we’re wrong about our thresholds, we’re still probably pretty close.”

Here’s what the half-cauchy prior model looks like for the variation of offensive strength across teams:

You can see how heavy-tailed this prior model is: once you get to about 20, the distribution if very flat. The x axis is clipped at 30, but it keeps extending well past 30. How heavy-tailed is it really? If you take 4,000 draws from the prior model, these are the most extreme values you’ll see:

419,159?! There’s 9 draws over 1,000, and 115 samples over 100. These are very extreme parameter values! Our prior model is saying: the variance across teams is probably somewhere less than 10, but if it’s more than 10, it can be anything!

These extreme values are obviously unreasonable. To suppress the unreasonably large values, we can switch to the more moderately-tailed normal distribution:

You can see most of the density is contained in the same region between zero and ten. Above ten, the cauchy distribution is much flatter, however.

Now let’s run our old formula: what is a reasonable bound on the offensive team strength variance parameter? Once we define those bounds, we can put 99% of our moderately-tailed normal distribution between those bounds. Unfortunately, this makes it obvious when this plug-and-chug formula breaks down. Unlike home field advantage, which everyone has intuition about, this is a more abstract parameter that is harder to pin down with your gut. In the next post, we’ll figure out what to do about that.

The Stan Model

To get all the prior distributions, I moved the entire prior model into the generated quantities block:

// Heirarchical IRT regression
//
// This models the points of home and away teams
// as a function of the latent offensive and defensive 
// strength of the teams.
//
// Specifically, this version tries to improve on the
// prior modeling.

data {
    // Number of games
    int<lower=1> N_games;

    // Number of teams in the league
    int<lower=1> N_teams;

    // Home and away points scored in each game
    array[N_games] int<lower=0> home_points;
    array[N_games] int<lower=0> away_points;

    // Team index for each game
    array[N_games] int<lower=1, upper=N_teams> home_team;
    array[N_games] int<lower=1, upper=N_teams> away_team;

    // Threshold for home field advantage
    // 99% of the density should be between 0 and this value
    real home_field_advantage_threshold;
}

transformed data {
    // Section 3.3.1 https://betanalpha.github.io/assets/case_studies/prior_modeling.html
    real home_field_advantage_prior_sigma = home_field_advantage_threshold / 2.57;
}


generated quantities {
   
    // Prior Modeling

    // Average strength of the teams
    real theta_offense_bar;
    theta_offense_bar = normal_rng(116, 10);

    // Home field advantage
    // Put 99% of dennsity between 0 and input {home_field_advantage_threshold}
    real <lower=0> home_field_advantage;
    home_field_advantage = abs(normal_rng(0, home_field_advantage_prior_sigma));

    // Variations of the teams strength
    real<lower=0> sigma_offense_bar;
    real<lower=0> sigma_defense_bar;   
    sigma_offense_bar = abs(cauchy_rng(0, 5));
    sigma_defense_bar = abs(cauchy_rng(0, 5));

    // Individual team strength
    real theta_offense;
    real theta_defense;    
    theta_offense = normal_rng(theta_offense_bar, sigma_offense_bar);
    theta_defense = normal_rng(0, sigma_defense_bar);

    // Gaussian noise in the points
    real<lower=0> sigma_points;
    sigma_points = abs(cauchy_rng(0, 5));

    real home_points_regression = home_field_advantage + theta_offense + theta_defense;
    real away_points_regression = theta_offense + theta_defense;
    real home_points_sample = normal_rng(home_points_regression, sigma_points);
    real away_points_sample = normal_rng(away_points_regression, sigma_points); 

}

As always, our full model is at the bottom of this post.

Home Field Advantage

Our original model had a home field advantage parameter baked into it. Here is what the posterior of that parameter looked like:

Home field advantage is somewhere between 2 and 2.5 points this season. But there is a lot of variance in that estimate. Home field advantage might be as low as half a point or as high as 4 points.

But what prior did we put on that parameter?

// Home field advantage, about 2 points
home_field_advantage ~ normal(2, 2);

I thought: “eh, home field advantage is about two points, and if I put a large variance on it, I should be safe.” But what is that prior actually saying? Here’s what that prior distribution looks like:

There’s one glaring problem: do we really expect home court advantage to be negative? This prior gives a 15% probability that home field advantage is less than zero. There’s also a slightly less glaring problem: Do we really expect home field advantage to be greater than 5 points? This prior gives a 7% probability that home field advantage is greater than 5 points.

Let’s develop a prior that more closely aligns with our domain expertise. The easiest (and arguably best) way to develop a prior isn’t to ask: what values do we expect our parameter to be. Instead, ask: what values do we not expect our parameter to be? This amounts to finding two thresholds: an upper bound and a lower bound (we don’t expect our parameter value to fall outside of those thresholds). How do we determine those thresholds? I like the mindset from this post on prior modeling: “Once we've transitioned from "I mean it's possible" to "oh, no, that's ridiculous" we know we've passed a meaningful threshold”.

To me, the lower bound on the number of points home field advantage is worth before it becomes ridiculous is zero. I can buy there is no home court advantage, but I’m not ready to buy that it’s negative. The upper bound? I think anything above 5 points would be ridiculous.

To convert those thresholds into a prior model, we do two things:

1. For the lower threshold (0 points), we use a hard threshold. We’ll sample from a half-normal(0, sigma) to prevent negative values.

2. For the upper threshold (5 points), we’ll use a soft threshold. We’ll adjust the sigma in our half-normal(0, sigma) such that 99% of the prior density falls between 0 and 5 points. This doesn’t completely prevent home field advantage from being greater than 5, but it aligns with our expectation that it would be very unlikely.

This is our refined prior distribution:

I chose 5 points as the upper threshold for home field advantage. But what happens if you disagree? Below, I’m plotting the resulting home field advantage posterior distributions. You can that see if you think home court advantage is way more (20 points for illustration), it doesn’t really affect the posterior probability. If you think the threshold is lower (2 points for illustration) then it really affects our final model.

Comparing our refined prior to our original prior

This is our original prior compared to our refined prior. It’s nice to see no values below zero and unreasonably large values less likely.

How did our prior refinement affect our resulting posterior estimate of home field advantage? Not much. The extreme values are suppressed a bit, in favor of more mild values.

Model Predictions

As before, our model is pumping out score line predictions (with a now-principled, albeit basically unchanged, home field advantage estimate).

Looking ahead

We have all sorts of wonky priors. The other ones are trickier to model in a principled manner, but we’ll work through it.

Stan Model

// Heirarchical IRT regression
//
// This models the points of home and away teams
// as a function of the latent offensive and defensive 
// strength of the teams.
//
// Specifically, this version tries to improve on the
// prior modeling.

data {
    // Number of games
    int<lower=1> N_games;

    // Number of teams in the league
    int<lower=1> N_teams;

    // Home and away points scored in each game
    array[N_games] int<lower=0> home_points;
    array[N_games] int<lower=0> away_points;

    // Team index for each game
    array[N_games] int<lower=1, upper=N_teams> home_team;
    array[N_games] int<lower=1, upper=N_teams> away_team;

    // Threshold for home field advantage
    // 99% of the density should be between 0 and this value
    real home_field_advantage_threshold;
}

transformed data {
    // Section 3.3.1 https://betanalpha.github.io/assets/case_studies/prior_modeling.html
    real home_field_advantage_prior_sigma = home_field_advantage_threshold / 2.57;
}

parameters {
    // Latent offensive and defensive strength of each team
    // Hierarchical prior
    vector[N_teams] theta_offense;
    vector[N_teams] theta_defense;
    real theta_offense_bar;
    real theta_defense_bar;
    real<lower=0> sigma_offense_bar;
    real<lower=0> sigma_defense_bar;

    // Noise in the points (same for home and away teams)
    real<lower=0> sigma_points;

    // Home field advantage is extremely unlikely to be negative
    real <lower=0> home_field_advantage;
}

model {

    // Prior Modeling

    // Average strength of the teams
    theta_offense_bar ~ normal(116, 10);

    // Home field advantage
    // Put 99% of dennsity between 0 and input {home_field_advantage_threshold}
    home_field_advantage ~ normal(0, home_field_advantage_prior_sigma);

    // Variations of the teams strength
    sigma_offense_bar ~ cauchy(0, 5);
    sigma_defense_bar ~ cauchy(0, 5);

    // Individual team strength
    theta_offense ~ normal(theta_offense_bar, sigma_offense_bar);
    theta_defense ~ normal(0, sigma_defense_bar);

    // Gaussian noise in the points
    sigma_points ~ cauchy(0, 5);

    // Likelihood
    for(game in 1:N_games) {
        // Team points modeled as gaussian
        real home_points_regression = home_field_advantage + theta_offense[home_team[game]] + theta_defense[away_team[game]];
        real away_points_regression = theta_offense[away_team[game]] + theta_defense[home_team[game]];
        home_points[game] ~ normal(home_points_regression, sigma_points);
        away_points[game] ~ normal(away_points_regression, sigma_points);    
    }
}

generated quantities {

    // Remove the mean from the latent variables
    vector[N_teams] theta_defense_centered;

    for (i in 1:N_teams) {
        theta_defense_centered[i] = theta_defense[i] - mean(theta_defense);
    }

    vector[N_teams] theta_offense_centered;

    for (i in 1:N_teams) {
        theta_offense_centered[i] = theta_offense[i] - mean(theta_offense);
    }
}

The Plan

We’re going to start a long-running series where we take one model, interrogate it, and iteratively improve it. Although we typically prefer more obscure models like modeling how consistent players are, for this series, we’re going mainstream: predicting NBA final score lines. Why did we decide to model something fairly monotonous:

1. It’s complicated enough to keep us busy through 2025

2. Everyone is interested in this.

The Starting Model

Our starting model is extremely simple, with many flaws. It’s a hierarchical model where each team’s offense and defensive strengths are simultaneously learned and modeled hierarchically. As always, the full model code is at the bottom of the post.

In this post, we’ll show the model and what it’s outputting. In later posts, we’ll dive into the specific limitations and work on iteratively improving them.

One thing to note is that this model is predicting total regulation points. In the future, overtime might be incorporated. Surely this won’t cause confusion in later posts when we stop mentioning this detail.

Game Predictions

Since we have a fully Bayesian model, we can get point estimates (pun), but there is also uncertainty in every estimate. So we can get not just which team our model thinks will win, but their win probability. And not just how many points they are likely to win by, but: what’s the percent chance they win by 10 points? what’s the percent chance they win by 11 points? etc.

Predictions for tonight’s match-ups:

Bayesian Power Rankings

Although we have a deeply limited and flawed model, we can still get offensive and defensive power rankings out of it. As a reminder, everyone can publish power rankings, but we are the only outlet that puts error bars on our power rankings.

Offensive Power Rankings

Defensive Power Rankings

Look ahead

There’s enough low hanging fruit with this model to tackle. If there’s something in particular you hate about it, let me know and I can prioritize that.

The Model

// Heirarchical IRT regression
//
// This models the points of home and away teams
// as a function of the latent offensive and defensive 
// strength of the teams.

data {
    // Number of games
    int<lower=1> N_games;

    // Number of teams in the league
    int<lower=1> N_teams;

    // Home and away points scored in each game
    array[N_games] int<lower=0> home_points;
    array[N_games] int<lower=0> away_points;

    // Team index for each game
    array[N_games] int<lower=1, upper=N_teams> home_team;
    array[N_games] int<lower=1, upper=N_teams> away_team;
}

parameters {
    // Latent offensive and defensive strength of each team
    // Hierarchical prior
    vector[N_teams] theta_offense;
    vector[N_teams] theta_defense;
    real theta_offense_bar;
    real theta_defense_bar;
    real<lower=0> sigma_offense_bar;
    real<lower=0> sigma_defense_bar;

    // Noise in the points (same for home and away teams)
    real<lower=0> sigma_points;

    real home_field_advantage;
}

model {

    // Priors
    // Average strength of the teams
    theta_offense_bar ~ normal(116, 10);

    // Home field advantage, about 2 points
    home_field_advantage ~ normal(2, 2);

    // Variations of the teams strength
    sigma_offense_bar ~ cauchy(0, 5);
    sigma_defense_bar ~ cauchy(0, 5);

    // Individual team strength
    theta_offense ~ normal(theta_offense_bar, sigma_offense_bar);
    theta_defense ~ normal(0, sigma_defense_bar);

    // Gaussian noise in the points
    sigma_points ~ cauchy(0, 5);

    // Likelihood
    for(game in 1:N_games) {
        // Team points modeled as gaussian
        real home_points_regression = home_field_advantage + theta_offense[home_team[game]] + theta_defense[away_team[game]];
        real away_points_regression = theta_offense[away_team[game]] + theta_defense[home_team[game]];
        home_points[game] ~ normal(home_points_regression, sigma_points);
        away_points[game] ~ normal(away_points_regression, sigma_points);    
    }
}

generated quantities {

    // Remove the mean from the latent variables
    vector[N_teams] theta_defense_centered;

    for (i in 1:N_teams) {
        theta_defense_centered[i] = theta_defense[i] - mean(theta_defense);
    }

    vector[N_teams] theta_offense_centered;

    for (i in 1:N_teams) {
        theta_offense_centered[i] = theta_offense[i] - mean(theta_offense);
    }
}

Subscribe now

The Model

As always, the full model is at the bottom of the post if you want to dive in. But here’s how it works at a high level:

• 3PT shooting for each player (and season) is modeled as binomial

• Player shooting ability is modeled hierarchically. So when a player has only taken 5 attempts so far this season (and made all 5 of them), the model shrugs and says it won’t keep happening.

• The difference in player shooting ability between this year and last year is sampled directly in the generated quantities.

If you’ve read enough of these posts, you probably could have guessed the model structure by now.

Improving Players

We’ll walk through LeBron James as an example. How well is LeBron James shooting from three this year and last year? And how certain are we in those estimates?

There’s a few things to notice:

• It looks like he’s better this year than last. There is a bit of overlap between the season’s estimates, but it’s probably safe to say LeBron James is improving (we'll come back to this).

• The uncertainty in this year’s estimate is greater than the uncertainty in last year’s estimate. As usual, this makes sense; we have less data to go on this year, so it’s harder to have precision in our estimates.

• LeBron James’s actual 3PT% this season is 39.5%. Our hierarchical model gives him a slightly lower shooting ability. We don’t have enough data to think he’s going to keep shooting at 39.5% yet.

So, how much has he improved? And how certain are we?

He’s probably improved about 5 percentage points, but the uncertainty is pretty large. Maybe 2.5%, maybe 7.5%. But there is a pretty good chance he improved.

Here are the top players that look like they improved at 3PT shooting this season (and our certainty in that improvement).

What’s most interesting is how few players we can say with certainty are improving. Man, Jalen Smith though.

Declining Players

Now, let’s look at players that are worse this season.

Luke Kennard really stands out.

Looking Ahead

I want add two improvements to the model:

1. For each player, put a prior on this year’s ability using their last years ability. If we haven’t seen much from them, assume they’re the same as last year.

2. Use other stat improvements to improve the 3PT shooting improvement estimate. If a player has very clearly improved in free throw shooting, is that predictive that they improved in 3PT shooting (even if we haven’t seen very much 3PT shooting yet?)

I think I’ll tackle these in multiple steps/posts.

I didn’t realize substack’s spell check was broken until after I sent out my last email. Nothing like coming back from a 2 year hiatus looking drunk.

Model Code

// Year-over-year improvement model

data {
  int<lower=1> N_players;
  int<lower=0> player_attempts_last_year[N_players];
  int<lower=0> player_attempts_this_year[N_players];
  int<lower=0> player_successes_last_year[N_players];
  int<lower=0> player_successes_this_year[N_players];
}

parameters {
    vector[N_players] theta_last_year;
    vector[N_players] theta_this_year;
    real theta_bar;
    real<lower=0> sigma_bar;
}

model {
    // Priors, hierarchical
    theta_bar ~ normal(-1, 10);
    sigma_bar ~cauchy(0, 5);
    theta_last_year ~ normal(theta_bar, sigma_bar);
    theta_this_year ~ normal(theta_bar, sigma_bar);

    // Likelihood
    for(player in 1:N_players) {
        player_successes_last_year[player] ~ binomial_logit(player_attempts_last_year[player], theta_last_year[player]);
        player_successes_this_year[player] ~ binomial_logit(player_attempts_this_year[player], theta_this_year[player]);
    }
}

generated quantities {
    // Transform theta back to probability
    vector[N_players] theta_last_year_probability;
    vector[N_players] theta_this_year_probability;
    theta_last_year_probability = inv_logit(theta_last_year);
    theta_this_year_probability = inv_logit(theta_this_year);

    // Calculate Difference
    vector[N_players] theta_probability_difference;
    theta_probability_difference = theta_this_year_probability - theta_last_year_probability;
}

Having taken a long hiatus, we thought to come back with a mundane question and simple models. Understanding whether or not a game goes into overtime has implications for league parity, corruption, and long-shot over bets.

Comparing this season to the last 24 years

What’s the probability that a random game from the last 24 seasons goes into overtime? Answer that question in your head and see how close you are.

One way to answer the question is calculate the fraction of games that have gone into overtime. But this is binomial basketball, so we fit a Bayesian binomial model and looked at the distribution of the rate parameter to get a sense not only of the probability, but our uncertainty in that estimate. As usual, all Stan models are at the end of the post.

There’s about a 5.9% chance that a random game in the last 24 years went into overtime. And there’s just so much data that there’s not much uncertainty. Probably somewhere between 5.6% and 6.1%.

But what about just this season? What if you wanted to estimate the chance that a random game coming up will go into overtime?

Two things to note:

1. This season looks less likely to go into overtime compared to a random game from the last 24 seasons. Around a 4.4% chance this season, compared to 5.9%

2. This season’s estimate has much less certainty. The estimate is probabilty somewhere between 3.5% and 5.5%. This makes sense, considering we have a lot less data when we look at just this season.

Hierarchical Binomial Model

There really hasn’t been much overtime this season to learn from. Using a hierarchical model, we can leverage information from other seasons to get a more accurate estimate of this season’s overtime probability.

Our hierarchical model puts a higher probability on a game this season going into overtime. Essentially, the model is saying: this season looks like an outlier so far and there’s not much data to go on, so probably the overtime probability is a touch higher than what we’ve seen so far.

And since we have the fully Bayesian hierarchical model, we can look at overtime probability over the years (overtime over time, if you will).

The obvious thing to notice is that since 2000, overtime probability has been trending downward, almost linearly. An obvious next step would be to regress on years within our hierarchical model.

Two questions for you:

1. Why do you think overtime probability is decreasing?

2. What was your a priori guess for the probability a game went into overtime?

Subscribe now

Looking ahead

I basically just pulled up my old code, scraped some data and got the Stan engine greased up again. The content of this post was not very interesting, but I’m thinking about getting into the groove again.

I’m going to move to the other side of Brooklyn soon.

Models

Simple Binomial Model

# Simple Binomial Model

data {
  // Number of Games
  int<lower=1> N_games;

  // Binary, overtime or not
  int<lower=0, upper=N_games> N_overtimes;
}

parameters {
    // Single binomial parameter
    real theta;
}

model {
    // Weak, uninformative prior
    theta ~ normal(-1, 10);

    // Number of overtimes is modeled binomial
    N_overtimes ~ binomial_logit(N_games, theta);
}

generated quantities {
    // Transform theta back to probability
    real theta_probability;
    theta_probability = inv_logit(theta);
}

Hierarchical Seasonal Binomial Model

# Hierarchical Binomial Regression

data {
  int<lower=1> N_seasons;
  int<lower=0> season_games[N_seasons];
  int<lower=0> season_overtimes[N_seasons];
}

parameters {
    vector[N_seasons] theta;
    real theta_bar;
    real<lower=0> sigma_bar;
}

model {
    theta_bar ~ normal(-1, 10);
    sigma_bar ~cauchy(0, 5);
    theta ~ normal(theta_bar, sigma_bar);
    for(season in 1:N_seasons) {
        season_overtimes[season] ~ binomial_logit(season_games[season], theta[season]);
    }
}

Percentiles are a generally useful way to think about a stat. They instantly give context to a number. Is a player scoring 1.0 points per possession on isolation shots good in isolation? That’s 76th percentile. There are two issues with using percentiles though:

1. Percentiles disregard the distribution of the statistic. For example, if nearly every player in the league scores between 0.99-1.01 points per possession on isolation plays, then the difference between the 20th and 80th percentile doesn’t matter. But maybe a few players score 1.4 points per possession on isolation plays. Then the difference between 95th and 98th percentiles is huge. If you only look at the percentile, these subtleties are lost.

2. They’re noisy.

Here, we’re going to focus on how percentiles are a noisy metric.

If you look at who’s the highest percentile on isolation shots on NBA.com (screenshot above), it’s Payton Pritchard. Jordan Nwora comes in second. That can’t be right.

Percentiles are exceptionally noisy for two reasons:

1. Any individual player has only taken a handful of isolation shots, so it’s hard to get a reasonable estimate of how good a player is at isolation shots. NBA.com uses a threshold of at least 10 isolation shots per player, but this is hacky and doesn’t really help.

2. Even if you could get a perfect estimate on a single player’s isolation shot ability, percentile is a relative metric. So let’s say you knew LeBron James made exactly 1.1 points per possession on isolation shots. It’s still hard to put a precise percentile on him when there’s so much uncertainty on how well every one else is shooting on isolation shots.

We used a model that estimates player isolation shot ability to try to get an understanding of how much uncertainty there is in percentile values. This allows us to put confidence intervals1 on the percentile value for each player.

The full model is at the bottom of this post, but it estimates each player’s points per possession on isolation shots using a hierarchical Poisson regression model that accounts for how many shot attempts each player has played. So if a player has only taken 5 isolation shot attempts and made all 5 of them, it looks around the league and says that the player probably got lucky. But if a player made 28 out of 30, that’s probably enough evidence that the player is really good in isolation.

Let’s start by comparing a couple players to get a sense of what the model is doing.

NBA.com has DeMar DeRozan at 91st percentile on 241 isolation possessions. Kevin Love is at 13th percentile on 26 isolation possessions. If you look at our model’s estimates of DeRozan’s and Love’s percentiles, it tells a slightly different story:

There’s so much data on DeMar DeRozan in isolation, that the model is fairly certain he’s >90th percentile. He’s probably even >95th percentile. Kevin Love, on the other hand, has only taken a handful of isolation shots. So really, he could be anywhere from the worst in the league to around 80th percentile. It’s just hard to say on only 26 isolation possessions.

Here’s another interesting example: Julius Randle has taken 272 isolation shots this season. That’s a lot, but here’s what the model thinks about his percentile:

The model is more certain about Randle’s percentile than Love’s, but there’s still huge uncertainty in Randle’s percentile. What’s going on here? The model is actually fairly certain in Randle’s isolation ability because there’s so much data on him. But there are so many other players with just a handful of isolation shots that the model doesn’t know where to put him. For example, Kevin Love might be better than Randle, but we just don’t know because we haven’t seen enough from Kevin Love this season.

The takeaway is that a single percentile value for each player in incredible noisy.

Looking across the league, our model gives a percentile estimate and confidence intervals for each player. Here are our model’s highest and lower percentile players.

Even at the upper and lower extremes, those confidence intervals are pretty dramatic.

Percentiles are useful to provide context to a stat, but it’s good to keep in mind the uncertainty in their values.

Looking Ahead

I spent so much time thinking about modeling percentiles, that I shouldn’t move on so quickly after just looking at isolation plays. But at the same time, I have another idea I’ve been meaning to look at:

Model

Here’s the full model. It’s a hierarchical Poisson model. I had a persistent bug while working on it, and while trying to fix the bug I ended up making a variety of improvements. For example, I used poisson_log directly instead of coding it myself. And I made the hierarchical parameter’s mean 0.

data {
    int<lower=0> n_players;
    int n_poss[n_players];
    int n_points[n_players];
    int y;
}

parameters {
    real lambda[n_players];
    real alpha;
    real<lower=0> lambda_sigma;
}

model {
    lambda_sigma ~ cauchy(0, 5);
    alpha ~ normal(0, 10);
    
    for (player in 1:n_players) {
        lambda[player] ~ normal(0, lambda_sigma);
        n_points[player] ~ poisson_log(log(n_poss[player]) + alpha + lambda[player]);
    }
}

generated quantities {
   int y_hat;
   int n_points_hat[n_players];
   real ppp[n_players];

   y_hat = 5;
   for (player in 1:n_players) {
        n_points_hat[player] = poisson_log_rng(log(n_poss[player]) + alpha + lambda[player]);
        ppp[player] = n_points_hat[player] * 1.0 / n_poss[player];
   }
}

In the following plots, we’re showing how inconsistent our model thinks a handful of players are for each stat. Players at the top of the plot are consistent game-to-game, and players at the bottom of the plot are less consistent (more fun to watch). Importantly, the confidence intervals show how certain the model is about it’s value for the players’ inconsistency.

We’ll give some general notes at the bottom, but there’s a lot to unpack here.

As we said previously: although the value in the plots has a real-life meaning related to how much variance each player is predicted to have game-to-game, it’s easiest just to think of this as Inconsistency Power Rankings.

The stat-of-interest is in the title, starting with points:

Some general notes:

• We included the “Steals” plot to make a point. Sometimes, you can’t draw any conclusions from a model. Look at the uncertainty in those values. You really can’t make any conclusive statements about how consistent players are in getting steals.

• In contrast to steals, you can make a pretty strong statement about Kevin Durant’s inconsistency in FGA.

• Some players are consistently consistent. Karl-Anthony Towns is basically the most consistent player at getting to the free throw line, rebounding, and scoring.

• As players get more inconsistent, the model has a harder time understanding precisely how inconsistent they are.

Looking Ahead

I’m going biking for a week or so in Utah. If anyone has recommendations, please let me know.

Model

For details about the model, see the previous post. It’s a hierarchical negative binomial regression.

data {
   
    int<lower=0> n_players;
    int<lower=0> n_stats;
    int stats[n_stats];
    int stats_player_index[n_stats];

}

parameters {

    // Mu terms
    vector<lower=0>[n_players] player_value;

    // Hierarchical Parameters
    real<lower=0> theta_bar;
    real<lower=0> sigma_bar;

    // Variance Terms
    vector<lower=0>[n_players] overdispersion;

}


model {

    // Hierarchical Prior on mu
    theta_bar ~ normal(0, 25);
    sigma_bar ~ cauchy(0, 5);
    player_value ~ normal(theta_bar, sigma_bar);

    //  Prior on Variance
    overdispersion ~ gamma(2, 2);


    // Negative Binomial Regression

    for (stat in 1:n_stats) {
      stats[stat] ~ neg_binomial_2(player_value[stats_player_index[stat]], overdispersion[stats_player_index[stat]]);
  }
}

generated quantities {
   int player_36_estimate[n_players];

   vector[n_players] player_variance;

   for (player in 1:n_players) {
     player_36_estimate[player] = neg_binomial_2_rng(player_value[player], overdispersion[player]);

     player_variance[player] = player_value[player] * player_value[player] / overdispersion[player];
   }

}