Concept map / Equating
Share
Equating alternate test forms
Use a common anchor to separate a difference in form difficulty from a difference in group ability, and compare Tucker and chained linear equating.
1Two forms, two groups, a 6-point gap
Group A takes Form A of a 50-item test and Group B takes Form B. Group B averages 6 points lower. Is Form B harder, or is Group B less able? Equating separates the two, so that a score means the same thing whichever form a student took.
When the groups were not assigned at random, both also answer the same 24 anchor items. Their gap on the anchor estimates how far apart the groups are. The green curve is Group B after the conversion. A correct conversion lands it on the gold curve, the scores Group B would have had on Form A.
Group B averages 6.00 points below Group A on the total score and 1.50 points below on the anchor. Chained linear adds +3.12 points to every Form B score; the true conversion is +3.00, so the method is off by +0.12.
All three methods side by side
Points added to every Form B score, and how far each method lands from the truth.
| Method | k | Conversion | Error |
|---|---|---|---|
| Ignore the anchor | 0.00 | +6.00 | +3.00 |
| Tucker | 1.47 | +3.79 | +0.79 |
| Chained linear | 1.92 | +3.12 | +0.12 |
2Predict how much of the gap is the form
Group A takes Form A and Group B takes Form B. Group B averages 6 points lower on its form than Group A does on theirs. On the shared 24-item anchor, Group B averages 1.5 points lower. How many points should be added to every Form B score?
3One formula, three choices of k
Every method starts from the gap in total scores and takes away the part the anchor puts down to the groups. The slope k converts one anchor point into total-score points. Both forms have the same spread here, so each conversion is a constant shift.
Now, Chained linear: (25.00 − 19.00) + 1.92 × (10.50 − 12.00) = +3.12 points.
Where each k comes from
Tucker uses the regression slope k = cov(X, V) / var(V) = 1.47. Chained linear uses the ratio of standard deviations, k = SD(X) / SD(V) = 1.92. The k that recovers the truth is 2.00, the ratio of how strongly ability moves the total and the anchor.
Error in the anchor scores pulls Tucker’s slope toward zero. Chained linear comes closer because error in the total partly cancels error in the anchor.
A common mistake is to trust Tucker because it is exact when the groups are equal. At δ = 0 the anchor gap is zero, so k multiplies nothing. At the starting δ of −0.5 it already adds 0.79 points too many, and the error grows with the group difference.
4Check yourself
The tasks use the buttons and sliders in section 1 and tick themselves when you meet them.
- Keep Form B harder than Form A, and find a group difference where ignoring the anchor would lower Form B scores instead of raising them.Now: Form B 3.0 points harder, ignoring the anchor adds +6.00
- With no anchor drift, find a setting where Tucker misses the truth by more than 1 point while chained linear misses by less than 0.25.Now: drift 0.00, Tucker off by 0.79, chained off by 0.12
- Give Group B an anchor drift and find a setting where the chained conversion misses the truth by more than 2 points.Now: drift 0.00, chained off by 0.12
5Tucker and chained equating with equate
The equate package works from frequency tables of total and anchor scores. The snippet simulates the lab’s starting setup with 20,000 students per group and puts Form B on the Form A scale three ways. It prints shifts of about 6.0, 3.9 and 3.2 points; the lab gives 6.00, 3.79 and 3.12. Rounding the simulated scores to whole points accounts for the difference.
Tucker and chained linear equating with an anchorShow the code
library(equate)
set.seed(1)
# One group: ability, a 50-item total, a 24-item anchor
make <- function(n, shift, harder) {
ability <- rnorm(n, mean = shift)
total <- 25 - harder + 6 * ability + rnorm(n, sd = 3)
anchor <- 12 + 3 * ability + rnorm(n, sd = 1.8)
cbind(total = pmin(50, pmax(0, round(total))),
anchor = pmin(24, pmax(0, round(anchor))))
}
# Group A takes Form A. Group B is 0.5 SD less able and
# takes Form B, which is 3 points harder.
a <- freqtab(make(20000, 0, 0), scales = list(0:50, 0:24))
b <- freqtab(make(20000, -0.5, 3), scales = list(0:50, 0:24))
# Put Form B scores on the Form A scale (true shift: +3)
none <- equate(b, a, type = "mean")
tuck <- equate(b, a, type = "linear", method = "tucker")
chain <- equate(b, a, type = "linear", method = "chained")
at25 <- function(e) e$concordance$yx[26] - 25
round(c(ignore_anchor = at25(none), tucker = at25(tuck),
chained = at25(chain)), 2)Uses equate. Install once with install.packages(c("equate")).
What equating can and cannot fix
- After equating, a score of 30 on Form B stands for the same performance as a 30 on Form A. The two groups can still differ, and equating leaves that difference in place.
- Without an anchor or random assignment, a gap between groups who took different forms cannot be split into form and group.
- Each anchor method subtracts k times the anchor gap from the raw gap, and the methods differ only in k. Tucker’s k is a regression slope, which error in the anchor shrinks, so it under-corrects when the groups are far apart.
- If the anchor items get easier for one group, through leaked items or changed teaching, that group looks more able than it is, and every anchor method blames too much of the gap on the form.
- Tests built to different blueprints cannot be equated, only linked, and the linking method changes what a converted score means.
Recommended reading
- Test Equating, Scaling, and LinkingMichael J. Kolen and Robert L. Brennan3rd ed., Nonequivalent Groups: Linear Methods
- Scaling, Equating, and LinkingTim MosesEducational Measurement, 5th ed., chapter 11