Concept map / Rasch and 2PL models

Share

Rasch and 2PL models

Fit a Rasch model and a 2PL to the same responses, decide between them, and see why the Rasch model scores by number correct.

1One common slope or one per item

Calibration estimates item parameters from responses. The Rasch model gives every item the same slope, so items differ only in difficulty and their curves never cross. The two-parameter logistic (2PL) model gives each item its own slope, or discrimination. A Rasch model is a 2PL with the slopes held equal, so a likelihood ratio test can compare the two.

Six items with difficulties from −1.5 to 1.5 are answered by simulated people whose ability θ is standard normal. Spread the true slopes apart. How much spread does it take before the data pay for five extra slope parameters?

People
Rasch: one common slope00.51−4−2024Ability θitem 4: a = 0.952PL: a slope for each item00.51−4−2024Ability θitem 4: a = 1.40
True curve of the chosen itemRasch fit2PL fit
Green marks the lower, preferred AIC and BIC.
FitRasch2PL
Parameters712
Log-likelihood−3595.41−3591.61
AIC7204.87207.2
BIC7239.27266.1

The true slopes run from 0.82 to 1.22. The Rasch model fits one common slope, 0.95, and the 2PL fits slopes from 0.80 to 1.40. With 1000 people the 2PL gains 3.80 in log-likelihood, so G² = 7.59 on 5 df (p = 0.180). The test does not reject equal slopes. AIC and BIC both prefer the Rasch model.

More controls
Sample 1. A new sample draws fresh responses from the same true model.

2Guess which model fits better when the data are Rasch

Set the spread to zero. Every true slope is now 1, so the Rasch model is exactly the model that generated the data. Fit both models to 1000 simulated people. Which one has the higher log-likelihood?

3Why the Rasch model scores by number correct

Write the log-likelihood of one person's responses uj as a function of θ. Under the Rasch model the responses meet θ only through their plain sum. That makes the sum score a sufficient statistic: once you know it, the rest of the pattern says nothing more about θ. Under the 2PL they meet θ through a sum weighted by the slopes. Point at a symbol to see what it stands for.

Rasch:log⁡L(θ)=θ⋅a∑juj  +  ∑jdjuj  −  ∑jlog⁡ ⁣(1+eaθ+dj)2PL:log⁡L(θ)=θ⋅∑jajuj  +  ∑jdjuj  −  ∑jlog⁡ ⁣(1+eajθ+dj)G2=2 (ℓ2PL−ℓRasch),df=2J−(J+1)=J−1\begin{aligned} \text{Rasch:}\quad \log L(\theta) &= \theta \cdot a\htmlClass{term-s}{\textstyle\sum_j u_j} \;+\; \textstyle\sum_j d_j u_j \;-\; \textstyle\sum_j \log\!\bigl(1 + e^{a\theta + d_j}\bigr) \\[2pt] \text{2PL:}\quad \log L(\theta) &= \theta \cdot \htmlClass{term-w}{\textstyle\sum_j a_j u_j} \;+\; \textstyle\sum_j d_j u_j \;-\; \textstyle\sum_j \log\!\bigl(1 + e^{a_j\theta + d_j}\bigr) \\[2pt] \htmlClass{term-lr}{G^2} &= 2\,(\ell_{\text{2PL}} - \ell_{\text{Rasch}}), \qquad \mathit{df} = 2J - (J + 1) = J - 1 \end{aligned}
Rasch:log⁡L(θ)=θ⋅a∑juj  +  ∑jdjuj−  ∑jlog⁡ ⁣(1+eaθ+dj)2PL:log⁡L(θ)=θ⋅∑jajuj  +  ∑jdjuj−  ∑jlog⁡ ⁣(1+eajθ+dj)G2=2 (ℓ2PL−ℓRasch)df=2J−(J+1)=J−1\begin{aligned} &\text{Rasch:} \\ \log L(\theta) &= \theta \cdot a\htmlClass{term-s}{\textstyle\sum_j u_j} \;+\; \textstyle\sum_j d_j u_j \\ &\quad -\; \textstyle\sum_j \log\!\bigl(1 + e^{a\theta + d_j}\bigr) \\[6pt] &\text{2PL:} \\ \log L(\theta) &= \theta \cdot \htmlClass{term-w}{\textstyle\sum_j a_j u_j} \;+\; \textstyle\sum_j d_j u_j \\ &\quad -\; \textstyle\sum_j \log\!\bigl(1 + e^{a_j\theta + d_j}\bigr) \\[6pt] \htmlClass{term-lr}{G^2} &= 2\,(\ell_{\text{2PL}} - \ell_{\text{Rasch}}) \\ \mathit{df} &= 2J - (J + 1) = J - 1 \end{aligned}

Point at the plain sum, the weighted sum or G².

Ability estimate (EAP)−2−10120123456Sum score (number correct)
Rasch estimate2PL estimateOne dot per observed pattern; larger dot, more people
Sum score

Sum score 3 has 19 different patterns, and the Rasch model gives every one of them −0.02. The 2PL estimates run from −0.16 for 011001 to 0.16 for 100110, a span of 0.32. Patterns list items 1 to 6 in order, so a 1 in fourth place is a right answer on item 4, the item with the steepest true slope.

4Check yourself

Each task ticks itself when the lab meets its condition.

  • With 250 people, spread the true slopes until the likelihood ratio test rejects the Rasch model at .05. If a sample is unlucky, draw another under More controls.Now: N = 1000, spread 0.20, G² = 7.59, p = 0.180
  • Find a setting where the test rejects the Rasch model but BIC still prefers it.Now: p = 0.180; BIC 7239.2 for Rasch and 7266.1 for the 2PL
  • With 1000 or more people, find a sum score in step 3 where the 2PL estimates for different patterns span more than 0.50.Now: N = 1000, sum score 3: 19 observed patterns, 2PL span 0.32, Rasch span 0.00

5Compare the two models in mirt

The code simulates 1000 people with the spread at 0.50, fits both models with mirt and compares them with anova(). R draws its own sample, so its numbers differ a little from the lab’s. In this run G² = 23.048 on 5 df rejects the Rasch model. AIC prefers the 2PL (7264.360 against 7277.408), while BIC prefers the Rasch model (7311.763 against 7323.253). The two patterns with a sum score of 3 share one Rasch estimate, 0.05, but get 2PL estimates of −0.080 and 0.243.

Why mirt’s Rasch estimates are on a different scale

mirt’s itemtype = "Rasch" fixes every slope at 1 and estimates the standard deviation of ability instead. The lab fixes that standard deviation at 1 and estimates one common slope. These are the same model, with the same J + 1 = 7 parameters and the same log-likelihood. Only the scale of θ differs, so mirt’s Rasch estimates equal the lab’s multiplied by the common slope. On the lab’s own simulated data, mirt matches the lab’s log-likelihoods, slopes, difficulties and EAP estimates to at least three decimals.

Compare Rasch and 2PL fits with mirtShow the code
library(mirt)
set.seed(1)
# Six items; spread 0.5 sets the true slopes exp(0.5 * shape)
b <- c(-1.5, -0.9, -0.3, 0.3, 0.9, 1.5)
shape <- c(-1, 0.6, -0.2, 1, -0.6, 0.2)
a <- exp(0.5 * shape)
dat <- simdata(a = matrix(a), d = matrix(-a * b), N = 1000,
               itemtype = "2PL")

# Rasch: slopes fixed at 1, ability SD estimated (J + 1 = 7
# parameters). 2PL: a slope per item, SD 1 (2J = 12).
rasch <- mirt(dat, 1, itemtype = "Rasch", verbose = FALSE)
twopl <- mirt(dat, 1, itemtype = "2PL", verbose = FALSE)
anova(rasch, twopl)
round(coef(twopl, IRTpars = TRUE, simplify = TRUE)$items[, 1:2], 2)

# Two patterns with the same sum score of 3
pats <- rbind(c(1, 1, 1, 0, 0, 0), c(0, 0, 0, 1, 1, 1))
round(cbind(rasch = fscores(rasch, response.pattern = pats)[, 1],
            twopl = fscores(twopl, response.pattern = pats)[, 1]), 3)

Uses mirt. Install once with install.packages(c("mirt")).

What to take away