Concept map / Fixed and random effects
Share
Fixed and random effects
Compare one line, fixed school intercepts, random school intercepts, and random school slopes fitted to the same students.
1Should a three-student school get its own line?
Nine schools report test scores and hours of study, with 3 to 40 students each. You could fit one line to everyone and ignore the schools, or fit each school alone and draw School A’s line through three points. A mixed-effects model sits in between. Its fixed effects, such as the average slope, are estimated once for everyone. Its random effects are each school’s deviations from them, treated as draws from a normal distribution, so a school’s estimate borrows strength from the others.
Switch between the four models, pick a school and change its size. The green line lands between the school’s own students and the population line, the same pull the multilevel models page shows for school means.
In the random intercepts model, each school intercept is pulled toward the population intercept, and the slope is shared. School A has 3 students. Its fitted score at 5 hours is 44.8 and its slope is 2.36 points per hour. With λ = 0.54, the estimate keeps 54% of the school’s distance from the population intercept.
2Three students or forty: which school moves more?
School A has 3 students and School H has 40. Both have adjusted means below the population intercept (42.8 and 41.1 against 47.0). Under random intercepts, each estimate moves from the school’s own mean toward the population. How do the two pulls compare?
3The weight on a school's own data
The study-time slope β₁ is a fixed effect in every model, one number for all schools. A random effect such as u₀ⱼ is a school’s deviation, modeled as a draw from a normal distribution with variance τ₀². Its estimate puts weight λ on the school’s own mean and the rest on the population value. Point at a term to see it on the chart.
Here τ₀ = 5, σ = 8, and School A has n = 3, so λ = 25 / (25 + 64/3) = 0.540. Its mean adjusted to 5 hours is ỹ = 42.80, and the population intercept is 47.04, so the random-intercept estimate is 47.04 + 0.540 × (42.80 − 47.04) = 44.75. The lab fixes the spreads (τ₀ = 5 points, τ₁ = 1.2 points per hour for slopes, σ = 8 points) instead of estimating them.
4Check yourself
The tasks use the controls in section 1 and tick themselves when you meet them.
- Give School A the smallest number of students at which its random-intercept weight λ reaches 0.90.Now: School A has 3 students, λ = 0.540
- Under random slopes, find a school whose slope estimate is more than 2 points per hour from the slope of its own students alone.Now: Random intercepts, School A: random-slopes estimate minus own slope = −2.57
- Under random intercepts, find a school whose estimate at 5 hours is more than 1 point below its fixed-effect estimate.Now: Random intercepts, School A: fixed minus random intercept = −1.95
5Random intercepts and slopes in lme4
lme4::lmer fits the lab’s random-slopes model, and (1 + x || school) gives each school its own intercept and slope deviation, uncorrelated as in the lab. Unlike the lab, it estimates the spreads from the data. With only nine schools they come out at 4.03 and 0.73, against the 5 and 1.2 used to simulate. The last lines show the pull on School A: its own three students give a slope of 4.67 points per hour, and the random-slopes model puts it at 2.11, close to the fixed slope of 2.02.
Random intercepts and slopes with lme4Show the code
library(lme4)
set.seed(4)
# Nine schools of the page's sizes, hours centered at 5
n <- c(A = 3, B = 4, C = 6, D = 8, E = 12, F = 18, G = 25,
H = 40, I = 40)
d <- data.frame(school = factor(rep(names(n), n)))
d$hours <- runif(nrow(d), 0, 10)
u0 <- rnorm(9, sd = 5)
u1 <- rnorm(9, sd = 1.2)
d$score <- 50 + u0[d$school] +
(2 + u1[d$school]) * (d$hours - 5) + rnorm(nrow(d), sd = 8)
d$x <- d$hours - 5
# Random intercepts and slopes, uncorrelated as on the page
rs <- lmer(score ~ x + (1 + x || school), data = d)
round(fixef(rs), 2)
print(VarCorr(rs), comp = "Std.Dev.")
# School A: its own 3-student line against the pooled one
own <- coef(lm(score ~ x, data = d, subset = school == "A"))
pooled <- unlist(coef(rs)$school["A", ])
round(rbind(own = own, random_slopes = pooled), 2)Uses lme4. Install once with install.packages(c("lme4")).
What random effects do
- The average slope is one fixed number. Each school’s deviation from it is a random effect, described by a variance instead of being estimated freely.
- A random effect is partially pooled. The school’s estimate is a weighted average of its own data and the population value, with weight λ = τ² / (τ² + σ²/n) on its own mean.
- The weight grows with the number of students, so School A with 3 students moves about half way toward the population while School H with 40 barely moves.
- Fixed school intercepts use each school’s data alone. That is unbiased for each school but noisy for the small ones.
- The same pooling underlies generalizability theory, where persons, tasks and raters are random facets, and item response theory, where each person’s ability is pulled toward the population mean.
Recommended reading
- Data Analysis Using Regression and Multilevel/Hierarchical ModelsAndrew Gelman and Jennifer HillChapters 11–13, varying intercepts, slopes, and partial pooling