Concept map / IRT scale linking

Share

IRT scale linking and true-score equating

Put two separately calibrated forms on one ability scale through their anchor items, then convert number-correct scores through the test curves.

1Two calibrations, two ability scales

Form X is new and Form Y is the base form. Each was calibrated on its own group, with that group’s mean ability set to 0 and its SD to 1, so b = 0.5 means a different difficulty on each form. The two scales differ by a straight line, θY = A·θX + B, where B is the new group’s mean and A its SD on the base scale.

Eight anchor items appear on both forms. Their two sets of b’s fall on that line, so they tell you A and B. Set the true A and B, add estimation error, and let Item 8 drift, as if it had leaked before Form X went out.

−2−1012−3−2−10123b on Form Y (base scale)b on Form X (its own scale)
Anchor itemMean/sigmaStocking–LordTrue line
Linking constants from the 8 anchors
MethodAB
True1.200.50
Mean/sigma1.190.50
Mean/mean1.190.50
Haebara1.190.50
Stocking–Lord1.190.51

Mean/sigma puts A at 1.19 and Stocking–Lord at 1.19; the true A is 1.20. The anchor farthest from the Stocking–Lord line is Item 2, 0.13 off on the base scale; next is Item 5 at 0.06.

More controls

Each b estimate is off by SE times a fixed random draw, in its own form’s units, and each a by 0.7 × SE of its size. Item 8 stays on Form X either way; leaving it out only removes it from the linking.

2Why one moved item drags mean/sigma

Mean/sigma uses only the b’s. A is the ratio of their standard deviations, and a standard deviation squares each item’s distance from the mean, so one anchor far from the others moves it a lot.

A=σ(bY)σ(bX),B=bˉY−A bˉX\htmlClass{term-ms}{A} = \frac{\sigma(\htmlClass{term-pts}{b_Y})}{\sigma(\htmlClass{term-pts}{b_X})}, \qquad \htmlClass{term-ms}{B} = \htmlClass{term-pts}{\bar{b}_Y} - \htmlClass{term-ms}{A}\,\htmlClass{term-pts}{\bar{b}_X}
A=σ(bY)σ(bX)B=bˉY−A bˉX\begin{aligned} \htmlClass{term-ms}{A} &= \frac{\sigma(\htmlClass{term-pts}{b_Y})}{\sigma(\htmlClass{term-pts}{b_X})} \\ \htmlClass{term-ms}{B} &= \htmlClass{term-pts}{\bar{b}_Y} - \htmlClass{term-ms}{A}\,\htmlClass{term-pts}{\bar{b}_X} \end{aligned}

Now: A = 1.080 / 0.908 = 1.19, and B = 0.102 − 1.19 × −0.335 = 0.50.

SL(A,B)=∑qwq[τY(θq)−τX∗(θq)]2τX∗(θ)=∑jP(θ; aXj/A,  A bXj+B)\begin{aligned} \mathrm{SL}(\htmlClass{term-sl}{A}, \htmlClass{term-sl}{B}) &= \textstyle\sum_q w_q \big[ \htmlClass{term-pts}{\tau_Y(\theta_q)} - \htmlClass{term-pts}{\tau^*_X(\theta_q)} \big]^2 \\ \tau^*_X(\theta) &= \textstyle\sum_j P(\theta;\, a_{Xj}/\htmlClass{term-sl}{A},\; \htmlClass{term-sl}{A}\, b_{Xj} + \htmlClass{term-sl}{B}) \end{aligned}

Stocking–Lord picks the A and B that bring τ*X, Form X’s anchor test curve after conversion, closest to τY, Form Y’s. The θq and wq are 30 points and weights for a standard normal group on the base scale. Now the two curves differ by a root mean square of 0.004 points at the Stocking–Lord constants and 0.018 at mean/sigma’s. Point at a symbol to light up its part of the chart in section 1.

Item 8 is the hardest anchor and has the flattest curve (a = 0.7, b = 1.7). Few examinees sit up there, so moving it changes the anchor test curve little where the weights are large. Haebara compares the curves item by item and holds up for the same reason. Mean/mean takes A from the a’s, so a drift in b moves only its B.

The Haebara and mean/mean formulas
H(A,B)=∑qwq∑j[PYj(θq)−PXj∗(θq)]2\mathrm{H}(A, B) = \textstyle\sum_q w_q \sum_j \big[ P_{Yj}(\theta_q) - P^*_{Xj}(\theta_q) \big]^2
A=aˉXaˉY,B=bˉY−A bˉXA = \frac{\bar{a}_X}{\bar{a}_Y}, \qquad B = \bar{b}_Y - A\,\bar{b}_X

Haebara squares the gap item by item, with P*Xj the converted Form X curve of item j. Both curve criteria here are minimized on the base scale only, as equateIRT does by default. Symmetric versions add the same sum on the Form X scale.

To move a Form X item onto the base scale, use a / A and A·b + B. A common slip is to multiply a by A. Stretching the ability scale by A spreads every curve out, so the slopes shrink by the same factor.

3A form that is harder everywhere

Build a Form X from Form Y’s 20 items by making every item 0.30 harder on the common scale, with the same discriminations. True-score equating then converts each Form X score to a Form Y score. How many points does that add?

4From one scale to equated scores

With Form X’s items on the base scale, IRT true-score equating needs no score distributions. For a number-correct score x on Form X, find the ability θ* where Form X’s test characteristic curve equals x, then read Form Y’s curve at θ*. The classical anchor methods worked from the score means and SDs of two groups; here the group difference is already in A and B.

5101520−4−2024Expected number correctθ* = 0.34Ability θ on the base scale
Form X after Stocking–LordForm YScore 10 and its equivalent
τX∗(θ∗)=x,eY(x)=τY(θ∗)\htmlClass{term-tx}{\tau^*_X}(\theta^*) = x, \qquad e_Y(x) = \htmlClass{term-ty}{\tau_Y}(\theta^*)
Conversion with Stocking–Lord constants
Form X scoreθ*Form Y equivalentTrue
4−1.354.784.73
8−0.178.928.88
120.8512.8712.85
162.0616.6316.63

Form X’s curve reaches 10 at θ* = 0.34. Form Y’s curve gives 10.92 there, so 10 on Form X counts as 10.92 on Form Y. The true equivalent is 10.89, so the Stocking–Lord constants are off by 0.03 points here.

With a guessing parameter (3PL)

Under the 3PL a test curve never drops below the sum of the guessing parameters, Σc. A Form X score below ΣcX has no θ* at all. The usual fix, described by Kolen and Brennan, draws a straight line from (0, 0) to (ΣcX, ΣcY) for those scores. The 2PL curves on this page start at 0, so every score from 1 to 19 has a θ*.

With the estimation error at 0 and no drift, every method recovers A and B exactly and the conversion matches the truth whatever A and B you set. Those two constants describe the calibrations, not the forms.

5Check yourself

The tasks use the controls in sections 1 and 4 and tick themselves when you meet them.

  • Make Item 8 drift until mean/sigma’s A misses the true A by more than 0.15 while Stocking–Lord’s stays within 0.05.Now: true A 1.20, mean/sigma 1.19, Stocking–Lord 1.19
  • Keep a drift of at least 1.0 either way, then leave Item 8 out of the anchor (under More controls) and bring all four values of A within 0.03 of the truth.Now: drift 0.0, Item 8 in the anchor, largest miss in A 0.01
  • In section 4, find a Form X score where the mean/sigma conversion misses the true Form Y equivalent by more than 1 point while Stocking–Lord misses it by less than 0.5.Now: Form X score 10: mean/sigma misses by 0.02, Stocking–Lord by 0.03

6Link and equate with equateIRT

The snippet builds the lab’s two forms with no estimation error, A = 1.2, B = 0.5 and Item 8 drifted 1.0 easier, so you can match it by setting SE to 0 and d to −1.0. direc() prints the four sets of constants: mean/sigma gives A = 1.413, the others 1.200 to 1.231. The last lines equate by hand with the Stocking–Lord constants and print 4.81, 9.08, 13.09 and 16.81 for Form X scores of 4, 8, 12 and 16, the same as the table in section 4 at that setting.

Link two 2PL calibrations and equate true scores with equateIRTShow the code
library(equateIRT)
# Form Y (base scale): 8 anchors, then 12 items of its own
a <- c(0.9, 1.3, 0.8, 1.1, 1.5, 1.0, 1.2, 0.7)
b <- c(-1.5, -1.0, -0.5, -0.1, 0.3, 0.7, 1.2, 1.7)
aY <- c(a, rep(c(1, 1.4, 0.8, 1.2), 3))
bY <- c(b, seq(-2, 2.2, length.out = 12))
# Form X on its own scale, theta_Y = 1.2 theta_X + 0.5.
# Item 8 has drifted from b = 1.7 to 0.7.
bX <- c(replace(b, 8, 0.7), seq(-1.6, 2.6, length.out = 12))
bX <- (bX - 0.5) / 1.2
aX <- c(a, rep(c(1.1, 0.9, 1.5, 1), 3)) * 1.2
X <- cbind(bX, aX)[1:8, ]
Y <- cbind(bY, aY)[1:8, ]
rownames(X) <- rownames(Y) <- paste0("Item", 1:8)
mods <- modIRT(coef = list(X, Y), names = c("X", "Y"),
               ltparam = FALSE, display = FALSE)
# Linking constants from the 8 anchors, four ways
AB <- sapply(c("mean-sigma", "mean-mean", "Haebara",
               "Stocking-Lord"), function(m) {
  l <- direc(mods, which = c("X", "Y"), method = m)
  c(A = l$A, B = l$B)
})
round(AB, 3)
# True-score equating with the Stocking-Lord constants
tcc <- function(th, a, b) sum(plogis(a * (th - b)))
A <- AB["A", 4]; B <- AB["B", 4]
toY <- function(x) {
  f <- function(t) tcc(t, aX / A, A * bX + B) - x
  tcc(uniroot(f, c(-9, 9), tol = 1e-10)$root, aY, bY)
}
round(sapply(c(4, 8, 12, 16), toY), 2)

Uses equateIRT. Install once with install.packages(c("equateIRT")).

What linking does