Concept map / Multidimensional IRT

Share

Multidimensional item response theory

See how an item’s response probability changes with two abilities, and interpret the direction and size of its discrimination.

1One item, two abilities

A math word problem draws on reading and on arithmetic. A multidimensional item response model gives the item a weight for each ability, a₁ and a₂, plus an intercept d. In the compensatory model used here, the chance of a correct answer depends only on the weighted sum a₁θ₁ + a₂θ₂ + d. Can a strong reader make up for weak arithmetic?

Each point on the floor of the surface is one combination of reading and arithmetic ability, and the height is the chance of a correct answer. The map shows the same item from above, with two people you can move.

Drag the picture to rotate it.

Height is the chance of a correct answer, from 0 on the floor to 1 at the top. The dark line marks 50%. The blue dot is person A and the small ring is person B.

More controls

This item leans on reading 2.0 times as much as arithmetic. The surface climbs fastest at 26.6° from the reading axis, and in that direction it is as steep as a one-ability item with discrimination 1.12. A person who is average in both abilities has a 38% chance.

2A strong reader who is weak at arithmetic

A math word problem needs reading and arithmetic equally: a₁ = 1, a₂ = 1 and d = 0. Person A reads very well but is weak at arithmetic (reading 2, arithmetic −2). Person B is average in both (0 and 0). Under the model on this page, who is more likely to answer correctly?

−3−3−2−2−1−100112233Arithmetic θ₂Reading θ₁BA

3Three numbers that summarize the item

The model is the 2PL curve from the item curve page with the single ability replaced by a weighted sum of two. Only that sum matters, so different mixes of ability can give the same chance. Point at a symbol to find it in the lab.

P(correct)  =  11+e−(a1 θ1  +  a2 θ2  +  d)P(\text{correct}) \;=\; \frac{1}{1 + e^{-(\htmlClass{term-a}{a_1}\,\htmlClass{term-b}{\theta_1} \;+\; \htmlClass{term-a}{a_2}\,\htmlClass{term-b}{\theta_2} \;+\; \htmlClass{term-c}{d})}}

For person A: 1.00 × 1.5 + 0.50 × (−1.0) + (−0.5) = 0.50, and 1 / (1 + e−0.50) = 0.622, or 62%.

The direction is the way the surface climbs fastest, the blend of abilities the item separates best. MDISC, the multidimensional discrimination, is the steepness in that direction and plays the part of a. The multidimensional difficulty is how far from the origin, along that direction, a person must stand for a 50% chance. It plays the part of b.

direction=arctan⁡a2a1MDISC=a12+a22difficulty=−dMDISC\text{direction} = \arctan\frac{\htmlClass{term-a}{a_2}}{\htmlClass{term-a}{a_1}} \qquad \text{MDISC} = \sqrt{\htmlClass{term-a}{a_1^2 + a_2^2}} \qquad \text{difficulty} = \frac{-\htmlClass{term-c}{d}}{\text{MDISC}}
direction=arctan⁡a2a1MDISC=a12+a22difficulty=−dMDISC\begin{aligned} \text{direction} &= \arctan\frac{\htmlClass{term-a}{a_2}}{\htmlClass{term-a}{a_1}} \\[4pt] \text{MDISC} &= \sqrt{\htmlClass{term-a}{a_1^2 + a_2^2}} \\[4pt] \text{difficulty} &= \frac{-\htmlClass{term-c}{d}}{\text{MDISC}} \end{aligned}

With your item: direction = arctan(0.50 / 1.00) = 26.6°. MDISC = √(1.00² + 0.50²) = 1.12. Difficulty = −(−0.5) / 1.12 = 0.45. It is positive, so a person who is average in both has less than a 50% chance.

A common mistake is to read d as the difficulty. It is an intercept: the same d gives a different difficulty once MDISC changes, because the difficulty is −d / MDISC.

Cut the surface along the arrow00.51−4−2024Chance of a correct answerPosition along the direction of best measurement (0 = the origin)
The surface, cut along the arrowSteepest slopePerson APerson B

Cut along the arrow, the surface is an ordinary 2PL curve with discrimination 1.12 and difficulty 0.45. Its steepest slope, MDISC / 4 = 0.280 per unit of ability, is on the 50% line. A sits at 0.89 and B at 0.00.

4Check yourself

The tasks use the first panel. Each ticks itself when you meet it.

  • Make an item that measures arithmetic only: direction 90°, with MDISC at least 0.5. Then look at which way the lines run on the map.Now: Direction 26.6°, MDISC 1.12
  • With an item that uses both abilities (a₁ and a₂ both above zero), place A and B on the map at least 2 units apart but on the same line of equal chance.Now: A 62%, B 38%, 2.8 units apart
  • Build a hard, sharp item that blends both abilities: direction between 35° and 55°, MDISC at least 1.5 and multidimensional difficulty at least 1.0.Now: Direction 26.6°, MDISC 1.12, difficulty 0.45

5A two-dimensional item in mirt

The code builds a two-dimensional model in mirt with the lab’s starting item fixed. probtrace gives persons A and B chances of 0.62 and 0.38. MDISC and MDIFF give 1.12 and 0.45, and the direction is 26.57° from the reading axis, which section 3 rounds to 26.6°.

A two-dimensional item in mirtShow the code
library(mirt)
# The page's item: a1 = 1 (reading), a2 = 0.5 (arithmetic),
# d = -0.5. Four copies give mirt a small test to work with.
set.seed(1)
dat <- simdata(matrix(c(1, 0.5), 4, 2, byrow = TRUE),
               rep(-0.5, 4), N = 500, itemtype = "2PL")
# A two-dimensional model with the parameters fixed
sv <- mirt(dat, 2, pars = "values")
sv$value[sv$name == "a1"] <- 1
sv$value[sv$name == "a2"] <- 0.5
sv$value[sv$name == "d"] <- -0.5
sv$est <- FALSE
mod <- mirt(dat, 2, pars = sv, verbose = FALSE)

# Chance of a correct answer for persons A and B
who <- rbind(A = c(1.5, -1), B = c(-0.5, 1))
round(probtrace(extract.item(mod, 1), who)[, 2], 2)
# Direction in degrees, MDISC and multidimensional difficulty
round(c(direction = atan2(0.5, 1) * 180 / pi,
        MDISC = unname(MDISC(mod)[1]),
        MDIFF = unname(MDIFF(mod)[1])), 2)

Uses mirt. Install once with install.packages(c("mirt")).

What the surface shows