Concept map / Test characteristic curve
Share
The test characteristic curve
Add response probabilities across items to find the expected number correct at an ability level.
1From item chances to a total score
Each item characteristic curve gives one student’s chance on one item, but a score report shows a total. A student with ability θ = 0.8 takes a six-item test. What total should we expect, and what changes on a harder version of the test?
The test characteristic curve (TCC) gives the expected number correct at each ability: the average total a student at that ability would earn over many sittings of the same form. Move θ, switch forms, and give the test to compare observed scores with the expected one.
- Item 1a 1.2, b −2.50.98
- Item 2a 0.9, b −1.60.90
- Item 3a 2.0, b −0.80.96
- Item 4a 0.8, b 0.30.60
- Item 5a 1.4, b 1.00.43
- Item 6a 1.3, b 2.10.16
- Sum4.02
At ability 0.8, the mixed form has an expected score of 4.02 out of 6. Give the test to see observed scores, which are always whole numbers. Moving θ or switching forms clears them.
2Predict where a gain in ability pays off
Two students take the mixed form. Ana’s ability rises from θ = 0 to θ = 1. Ben’s rises from θ = 2.5 to θ = 3.5. Both gain one unit of ability. Whose expected number correct rises more?
3Six item chances add up to the curve
An item scored 1 or 0 has an expected score equal to its chance of a correct answer, so the expected total is the sum of the six chances. Point at a symbol to light up its part of the lab.
Each Pi depends only on θ − bi. Add the same amount to every b, as the harder form does, and the whole curve slides right by that amount without changing shape.
4Check yourself
Each task ticks itself when the lab meets its condition.
- On the mixed form, find an ability where the expected score is half the test, 3 out of 6 (within 0.05).Now: Mixed form, θ = 0.8, expected score 4.02
- A student at θ = 0.8 expects 4.02 on the mixed form. On the harder form, find the ability that gives the same expected score (within 0.05).Now: Mixed form, θ = 0.8, expected score 4.02
- Give the test at least 20 times at one ability and form, and get the average observed score within 0.3 of the expected score. Moving θ or the form starts the count again.Now: no administrations at this setting yet
5The same curve by hand and with mirt
The code adds the six item chances at θ = 0.8 by hand, giving 4.02 as in the lab, and shows that the harder form gives the same expected score at θ = 2.0. It then fits the six items to simulated answers with mirt and reads the fitted TCC with expected.test(). The fitted curve gives 3.04, 4.07 and 5.05 at θ = −0.3, 0.8 and 2.0, close to the true values of 2.99, 4.02 and 5.02.
Compute a test characteristic curve by hand and with mirtShow the code
library(mirt)
# The six items of the mixed form (2PL, no 1.7 constant)
a <- c(1.2, 0.9, 2.0, 0.8, 1.4, 1.3)
b <- c(-2.5, -1.6, -0.8, 0.3, 1.0, 2.1)
p <- function(theta) 1 / (1 + exp(-a * (theta - b)))
# Expected number correct at theta = 0.8: add the chances
round(p(0.8), 2)
round(sum(p(0.8)), 2)
# The harder form adds 1.2 to every b, so theta = 2.0 there
# gives the same expected score
round(sum(1 / (1 + exp(-a * (2.0 - (b + 1.2))))), 2)
# Fit the six items to 3000 simulated students with mirt,
# then read the fitted curve at three abilities
set.seed(2)
resp <- simdata(matrix(a), matrix(-a * b), N = 3000,
itemtype = "2PL")
fit <- mirt(resp, 1, itemtype = "2PL", verbose = FALSE)
round(expected.test(fit, Theta = matrix(c(-0.3, 0.8, 2))), 2)Uses mirt. Install once with install.packages(c("mirt")).
What the curve says about scores
- For items scored 0 or 1, the expected number correct at θ is the sum of the item probabilities there. On the mixed form at θ = 0.8 that is 4.02 out of 6.
- One sitting gives a whole number that scatters around the expected score. The average over many sittings settles near the curve.
- The curve is steep where item difficulties cluster and flat near 0 and 6. In the prediction, one unit of ability is worth 0.91 points from θ = 0 to 1 and only 0.41 from θ = 2.5 to 3.5.
- Adding the same amount to every difficulty slides the TCC right by that amount. Because the curve always rises, each expected score matches one ability, so scores on different forms can be compared through θ. That is IRT true-score equating.
- The TCC says what score to expect, not how precisely the form measures. Precision is the job of test information.
Recommended reading
- The Basics of Item Response Theory Using RFrank B. Baker and Seock-Ho KimChapter 5, the test characteristic curve