Concept map / Equating

Share

Equating alternate test forms

Use a common anchor to separate a difference in form difficulty from a difference in group ability, and compare Tucker and chained linear equating.

1Two forms, two groups, a 6-point gap

Group A takes Form A of a 50-item test and Group B takes Form B. Group B averages 6 points lower. Is Form B harder, or is Group B less able? Equating separates the two, so that a score means the same thing whichever form a student took.

When the groups were not assigned at random, both also answer the same 24 anchor items. Their gap on the anchor estimates how far apart the groups are. The green curve is Group B after the conversion. A correct conversion lands it on the gold curve, the scores Group B would have had on Form A.

01020304050Share of students (density)Score on the Form A scale (0 to 50)conversion +3.12 (true +3.00)
Group A on Form AGroup B on Form B, rawGroup B after conversionGroup B if they had taken Form A
Mean on the shared 24-item anchor06121824A 12.00B 10.50

Group B averages 6.00 points below Group A on the total score and 1.50 points below on the anchor. Chained linear adds +3.12 points to every Form B score; the true conversion is +3.00, so the method is off by +0.12.

All three methods side by side

Points added to every Form B score, and how far each method lands from the truth.

MethodkConversionError
Ignore the anchor0.00+6.00+3.00
Tucker1.47+3.79+0.79
Chained linear1.92+3.12+0.12

2Predict how much of the gap is the form

Group A takes Form A and Group B takes Form B. Group B averages 6 points lower on its form than Group A does on theirs. On the shared 24-item anchor, Group B averages 1.5 points lower. How many points should be added to every Form B score?

3One formula, three choices of k

Every method starts from the gap in total scores and takes away the part the anchor puts down to the groups. The slope k converts one anchor point into total-score points. Both forms have the same spread here, so each conversion is a constant shift.

conversion=(XˉA−YˉB)+k (VˉB−VˉA)\text{conversion} = \htmlClass{term-gap}{(\bar X_A - \bar Y_B)} + \htmlClass{term-k}{k}\,\htmlClass{term-anchor}{(\bar V_B - \bar V_A)}

Now, Chained linear: (25.00 − 19.00) + 1.92 × (10.50 − 12.00) = +3.12 points.

Where each k comes from

Tucker uses the regression slope k = cov(X, V) / var(V) = 1.47. Chained linear uses the ratio of standard deviations, k = SD(X) / SD(V) = 1.92. The k that recovers the truth is 2.00, the ratio of how strongly ability moves the total and the anchor.

Error in the anchor scores pulls Tucker’s slope toward zero. Chained linear comes closer because error in the total partly cancels error in the anchor.

A common mistake is to trust Tucker because it is exact when the groups are equal. At δ = 0 the anchor gap is zero, so k multiplies nothing. At the starting δ of −0.5 it already adds 0.79 points too many, and the error grows with the group difference.

4Check yourself

The tasks use the buttons and sliders in section 1 and tick themselves when you meet them.

  • Keep Form B harder than Form A, and find a group difference where ignoring the anchor would lower Form B scores instead of raising them.Now: Form B 3.0 points harder, ignoring the anchor adds +6.00
  • With no anchor drift, find a setting where Tucker misses the truth by more than 1 point while chained linear misses by less than 0.25.Now: drift 0.00, Tucker off by 0.79, chained off by 0.12
  • Give Group B an anchor drift and find a setting where the chained conversion misses the truth by more than 2 points.Now: drift 0.00, chained off by 0.12

5Tucker and chained equating with equate

The equate package works from frequency tables of total and anchor scores. The snippet simulates the lab’s starting setup with 20,000 students per group and puts Form B on the Form A scale three ways. It prints shifts of about 6.0, 3.9 and 3.2 points; the lab gives 6.00, 3.79 and 3.12. Rounding the simulated scores to whole points accounts for the difference.

Tucker and chained linear equating with an anchorShow the code
library(equate)
set.seed(1)

# One group: ability, a 50-item total, a 24-item anchor
make <- function(n, shift, harder) {
  ability <- rnorm(n, mean = shift)
  total <- 25 - harder + 6 * ability + rnorm(n, sd = 3)
  anchor <- 12 + 3 * ability + rnorm(n, sd = 1.8)
  cbind(total = pmin(50, pmax(0, round(total))),
        anchor = pmin(24, pmax(0, round(anchor))))
}
# Group A takes Form A. Group B is 0.5 SD less able and
# takes Form B, which is 3 points harder.
a <- freqtab(make(20000, 0, 0), scales = list(0:50, 0:24))
b <- freqtab(make(20000, -0.5, 3), scales = list(0:50, 0:24))

# Put Form B scores on the Form A scale (true shift: +3)
none <- equate(b, a, type = "mean")
tuck <- equate(b, a, type = "linear", method = "tucker")
chain <- equate(b, a, type = "linear", method = "chained")
at25 <- function(e) e$concordance$yx[26] - 25
round(c(ignore_anchor = at25(none), tucker = at25(tuck),
        chained = at25(chain)), 2)

Uses equate. Install once with install.packages(c("equate")).

What equating can and cannot fix