Concept map / Item information

Share

Item information and measurement precision

Combine item information curves and compare approximate standard errors across abilities.

1Where on the scale does a test measure well?

An item tells you most about people whose ability is near its difficulty. Someone far above an easy item almost always gets it right, so that answer says little about them. Item information puts a number on this. For a 2PL item curve it is a² P(1 − P): the slope squared, times the chance of a right answer, times the chance of a wrong one.

Add items from the bank to the test. The upper chart stacks their information. The lower one shows the approximate standard error, the margin of error of an ability estimate, at each ability. Move θ to read both at one point.

5 of 14 items in the test
0123Information: each band is one item, stacked-4-3-2-101234Ability θ00.511.52+Approximate standard error: lower is better-4-3-2-101234Ability θ1.35

This 5-item test is most informative near θ = −0.2, where the standard error is 0.67. For a person at θ = 2.00 the information is 0.55 and the standard error is 1.35, 2.0 times as large.

2Which item helps a strong candidate most?

A five-item test has all its items near medium difficulty; the gray curve is its information. You need to measure a strong candidate at θ = 2 (the green line), where the approximate standard error is ± 1.33. You may add one item. Which one lowers that error most?

012-4-2024InformationAbility θ

3Slope squared times P(1 − P)

Each band in the stacked chart is one item's a² P(1 − P), and the standard error comes from their sum. Point at a part of the formula to see what it does.

I(θ)  =  ∑iai 2  Pi(θ) [1−Pi(θ)]SE(θ)  =  1I(θ)I(\theta) \;=\; \sum_i \htmlClass{term-a}{a_i^{\,2}}\;\htmlClass{term-pq}{P_i(\theta)\,\bigl[1-P_i(\theta)\bigr]} \qquad\quad \text{SE}(\theta) \;=\; \frac{1}{\sqrt{I(\theta)}}
I(θ)  =  ∑iai 2  Pi(θ) [1−Pi(θ)]SE(θ)  =  1I(θ)\begin{gathered} I(\theta) \;=\; \sum_i \htmlClass{term-a}{a_i^{\,2}}\;\htmlClass{term-pq}{P_i(\theta)\,\bigl[1-P_i(\theta)\bigr]} \\[6pt] \text{SE}(\theta) \;=\; \frac{1}{\sqrt{I(\theta)}} \end{gathered}

Point at the orange or the blue part of the formula.

At θ = 2.00: 0.01 + 0.08 + 0.10 + 0.10 + 0.26 = 0.55, so the standard error is 1 / √0.55 = 1.35.

4Check yourself: build a test for a purpose

Choosing items so the information sits where it is needed is called test assembly. Each task needs a different set of items from the bank above, and each ticks itself when the test meets its condition.

  • A pass/fail exam has its cut score at θ = 1. Using at most 4 items, get the standard error at θ = 1 to 0.70 or less.Now: 5 items, 0.79 at θ = 1
  • A screening test must identify people who are struggling. Using at most 4 items, find a test with standard error of 0.78 or less at θ = −2.Now: 5 items, 1.25 at θ = −2
  • A general-purpose test should work for everyone. Using at most 8 items, keep the standard error at 0.77 or less all the way from θ = −2 to θ = 2.Now: 5 items, largest standard error 1.35 between −2 and 2

5Item and test information in mirt

The code fixes the lab’s starting test, items 5 to 9, in mirt. iteminfo gives each item’s information at θ = 2, and testinfo adds them to 0.55, a standard error of 1.35, as the lab shows. At θ = 0 the same five items give 2.19 and 0.68.

Item and test information with mirtShow the code
library(mirt)
# The page's starting test: items 5 to 9 of the bank
a <- c(2.0, 1.0, 1.6, 0.8, 1.9)
b <- c(-0.8, -0.4, 0.0, 0.3, 0.7)
# Build a 2PL model with the parameters fixed at these values
set.seed(1)
dat <- simdata(a, -a * b, N = 500, itemtype = "2PL")
sv <- mirt(dat, 1, pars = "values")
sv$value[sv$name == "a1"] <- a
sv$value[sv$name == "d"] <- -a * b
sv$est <- FALSE
mod <- mirt(dat, 1, pars = sv, verbose = FALSE)

# Information from each item at theta = 2
round(sapply(1:5, function(i)
  iteminfo(extract.item(mod, i), matrix(2))), 2)
# Test information is their sum; SE = 1 / sqrt(information)
th <- matrix(c(0, 2))
info <- testinfo(mod, th)
round(cbind(theta = c(0, 2), info, se = 1 / sqrt(info)), 2)

Uses mirt. Install once with install.packages(c("mirt")).

How information sets precision