Concept map / Fixed and random effects

Share

Fixed and random effects

Compare one line, fixed school intercepts, random school intercepts, and random school slopes fitted to the same students.

1Should a three-student school get its own line?

Nine schools report test scores and hours of study, with 3 to 40 students each. You could fit one line to everyone and ignore the schools, or fit each school alone and draw School A’s line through three points. A mixed-effects model sits in between. Its fixed effects, such as the average slope, are estimated once for everyone. Its random effects are each school’s deviations from them, treated as draws from a normal distribution, so a school’s estimate borrows strength from the others.

Switch between the four models, pick a school and change its size. The green line lands between the school’s own students and the population line, the same pull the multilevel models page shows for school means.

10305070900246810Test scoreHours studied
Students in School ARandom intercepts fitPopulation line

In the random intercepts model, each school intercept is pulled toward the population intercept, and the slope is shared. School A has 3 students. Its fitted score at 5 hours is 44.8 and its slope is 2.36 points per hour. With λ = 0.54, the estimate keeps 54% of the school’s distance from the population intercept.

2Three students or forty: which school moves more?

School A has 3 students and School H has 40. Both have adjusted means below the population intercept (42.8 and 41.1 against 47.0). Under random intercepts, each estimate moves from the school’s own mean toward the population. How do the two pulls compare?

SchoolScore at 5 hours3841444750population 47.0A (n = 3)H (n = 40)SchoolScore at 5 hours3841444750population 47.0A (n = 3)H (n = 40)
School’s own adjusted mean

3The weight on a school's own data

The study-time slope β₁ is a fixed effect in every model, one number for all schools. A random effect such as u₀ⱼ is a school’s deviation, modeled as a draw from a normal distribution with variance τ₀². Its estimate puts weight λ on the school’s own mean and the rest on the population value. Point at a term to see it on the chart.

yij=(β0+u0j)+β1(xij−5)+eij,u0j∼N(0,τ02)y_{ij}=(\beta_0+\htmlClass{term-a}{u_{0j}})+\beta_1(x_{ij}-5)+e_{ij},\quad u_{0j}\sim N(0,\tau_0^2)
yij=(β0+u0j)+β1(xij−5)+eij,u0j∼N(0,τ02)\begin{gathered} y_{ij}=(\beta_0+\htmlClass{term-a}{u_{0j}})+\beta_1(x_{ij}-5)+e_{ij}, \\[4pt] u_{0j}\sim N(0,\tau_0^2) \end{gathered}
β^0+u^0j=β^0+λj (y~j−β^0),λj=τ02τ02+σ2/nj\hat\beta_0+\hat u_{0j} = \hat\beta_0 + \htmlClass{term-theta}{\lambda_j}\,(\htmlClass{term-a}{\tilde y_j}-\hat\beta_0),\qquad \htmlClass{term-theta}{\lambda_j}=\frac{\tau_0^2}{\tau_0^2+\sigma^2/n_j}
β^0+u^0j=β^0+λj (y~j−β^0),λj=τ02τ02+σ2/nj\begin{gathered} \hat\beta_0+\hat u_{0j} = \hat\beta_0 + \htmlClass{term-theta}{\lambda_j}\,(\htmlClass{term-a}{\tilde y_j}-\hat\beta_0), \\[6pt] \htmlClass{term-theta}{\lambda_j}=\frac{\tau_0^2}{\tau_0^2+\sigma^2/n_j} \end{gathered}

Here τ₀ = 5, σ = 8, and School A has n = 3, so λ = 25 / (25 + 64/3) = 0.540. Its mean adjusted to 5 hours is ỹ = 42.80, and the population intercept is 47.04, so the random-intercept estimate is 47.04 + 0.540 × (42.80 − 47.04) = 44.75. The lab fixes the spreads (τ₀ = 5 points, τ₁ = 1.2 points per hour for slopes, σ = 8 points) instead of estimating them.

4Check yourself

The tasks use the controls in section 1 and tick themselves when you meet them.

  • Give School A the smallest number of students at which its random-intercept weight λ reaches 0.90.Now: School A has 3 students, λ = 0.540
  • Under random slopes, find a school whose slope estimate is more than 2 points per hour from the slope of its own students alone.Now: Random intercepts, School A: random-slopes estimate minus own slope = −2.57
  • Under random intercepts, find a school whose estimate at 5 hours is more than 1 point below its fixed-effect estimate.Now: Random intercepts, School A: fixed minus random intercept = −1.95

5Random intercepts and slopes in lme4

lme4::lmer fits the lab’s random-slopes model, and (1 + x || school) gives each school its own intercept and slope deviation, uncorrelated as in the lab. Unlike the lab, it estimates the spreads from the data. With only nine schools they come out at 4.03 and 0.73, against the 5 and 1.2 used to simulate. The last lines show the pull on School A: its own three students give a slope of 4.67 points per hour, and the random-slopes model puts it at 2.11, close to the fixed slope of 2.02.

Random intercepts and slopes with lme4Show the code
library(lme4)
set.seed(4)
# Nine schools of the page's sizes, hours centered at 5
n <- c(A = 3, B = 4, C = 6, D = 8, E = 12, F = 18, G = 25,
       H = 40, I = 40)
d <- data.frame(school = factor(rep(names(n), n)))
d$hours <- runif(nrow(d), 0, 10)
u0 <- rnorm(9, sd = 5)
u1 <- rnorm(9, sd = 1.2)
d$score <- 50 + u0[d$school] +
  (2 + u1[d$school]) * (d$hours - 5) + rnorm(nrow(d), sd = 8)
d$x <- d$hours - 5

# Random intercepts and slopes, uncorrelated as on the page
rs <- lmer(score ~ x + (1 + x || school), data = d)
round(fixef(rs), 2)
print(VarCorr(rs), comp = "Std.Dev.")

# School A: its own 3-student line against the pooled one
own <- coef(lm(score ~ x, data = d, subset = school == "A"))
pooled <- unlist(coef(rs)$school["A", ])
round(rbind(own = own, random_slopes = pooled), 2)

Uses lme4. Install once with install.packages(c("lme4")).

What random effects do