Concept map / Polytomous IRT

Share

Polytomous item response models

Compare cumulative and category probabilities for an item scored in four ordered categories.

Prerequisites IRT: item curves

1Four scores, three thresholds

An essay scored 0, 1, 2 or 3 has three boundaries between neighboring scores. Samejima’s graded response model gives each boundary an ordinary item curve: the chance of scoring at or above it. The chance of one exact score is the gap between two of those curves. What happens to a score of 2 when the thresholds on either side of it move close together?

The buttons switch to the partial credit model, which reads the same three numbers as step difficulties: the abilities at which two neighboring scores are equally likely. Drag the threshold handles, or the green dot for the examinee.

Each threshold b is the ability at which the chance of reaching that score or higher is 50%.

00.250.50.751-4-3-2-101234Ability θP(exactly this score)b₁b₂b₃score 2: 47%
Score 0Score 1Score 2Score 3Two neighboring scores equally likely
More controls

The thresholds stay in order and at least 0.1 apart, so a slider stops at its neighbor.

An examinee at θ = 1.00 most likely scores 2, with a chance of 47%. Averaged over many examinees at this ability, the score would be 2.22 out of 3.

Where neighboring curves cross

scores 0 and 1 are equally likely at θ = −1.28 (b₁ is −1.50), scores 1 and 2 are equally likely at θ = −0.26 (b₂ is −0.20), and scores 2 and 3 are equally likely at θ = 1.14 (b₃ is 1.30). A graded threshold marks a 50% chance of reaching a score, so the crossings sit somewhere else.

2Two thresholds close together

Predict first. This essay item is scored 0 to 3, with a = 1.5 and thresholds at −1.5, 0 and 1.5. The orange hump is the chance of scoring exactly 2. It lives between the second and third thresholds. We are about to slide the third threshold down from 1.5 to 0.3, so that it sits right next to the second. What happens to the orange hump?

00.51-4-2024P(score)Ability θb₁b₂b₃
Score 0Score 1Score 2Score 3

Second and third thresholds 1.50 apart: score 2 peaks at 51%, at θ = 0.75.

3What the item measures, and where

012-4-3-2-101234Ability θInformationpeak 0.66 at θ = −0.41
This item, scored 0 to 3The same item scored right or wrong at b₂

At θ = 1.00 the item gives information of 0.63. It stays above 0.28, half the peak of the right/wrong version, over 5.2 units of ability. Scored right or wrong, it covers 2.4.

4A score is the gap between two curves

The graded model has no formula for a single score. It has 2PL curves for "this score or higher", written P*, and takes differences. Choose a score, then point at a term in the equation to light it up in the chart.

00.51-4-3-2-101234Ability θP(this score or higher)gap 0.469
P* for score 2: chance of scoring 2 or higherP* for score 3: chance of scoring 3 or higherThe gap, P(score = 2)
P(score=k)  =  Pk∗(θ)  −  Pk+1∗(θ)Pk∗(θ)  =  11+e−a (θ−bk)\begin{aligned} \htmlClass{term-polycat}{P(\text{score}=k)} \;&=\; \htmlClass{term-polyhi}{P^*_{k}(\theta)} \;-\; \htmlClass{term-polylo}{P^*_{k+1}(\theta)} \\[6pt] P^*_{k}(\theta) \;&=\; \frac{1}{1+e^{-a\,(\theta-b_k)}} \end{aligned}

At θ = 1.00: P(score = 2) = 0.858 − 0.389 = 0.469. Here P*2 = 1 / (1 + e−1.50(1.00 − (−0.20))) = 0.858. The four gaps run from the line at 1 down to the line at 0, so the four chances always add to 1.

5Check yourself

The tasks use the first panel. Each ticks itself when you meet it.

  • Squeeze a middle score out: move the thresholds until score 1 or score 2 is not the most likely score at any ability.Now: Most likely somewhere: scores 0, 1, 2, 3
  • Build an item and place an examinee so that their most likely score is 2, with a chance of at least 70%.Now: Examinee at θ = 1.00: most likely score 2, probability 46.8%
  • Get information of at least 0.60 at every ability from −2 to 2 (section 3 shows it). At a = 1.5 only the generalized partial credit model gets there; a is under More controls.Now: Lowest information between −2 and 2 is 0.43

6Graded and partial credit items in mirt

The code fixes the lab’s starting item in mirt twice: as a graded response item and as a partial credit item (itemtype = "Rasch") with the same three numbers as step difficulties. At θ = 1 the graded item has an expected score of 2.22 and information of 0.63. The partial credit item has 2.19 and 0.52. Both match the lab. For the generalized model, use itemtype = "gpcm".

Graded response and partial credit items in mirtShow the code
library(mirt)
# The page's item, scored 0 to 3
a <- 1.5
b <- c(-1.5, -0.2, 1.3)
set.seed(1)
d <- matrix(-a * b, 2, 3, byrow = TRUE)
dat <- simdata(matrix(a, 2), d, N = 500, itemtype = "graded")
# A model with the item parameters fixed, so nothing is fitted
fixed <- function(type, slope, d) {
  sv <- mirt(dat, 1, itemtype = type, pars = "values")
  sv$value[sv$name == "a1"] <- slope
  for (k in 1:3) sv$value[sv$name == paste0("d", k)] <- d[k]
  sv$est <- FALSE
  mirt(dat, 1, itemtype = type, pars = sv, verbose = FALSE)
}
# Graded response, then partial credit (a = 1, steps = b)
mods <- list(GRM = fixed("graded", a, -a * b),
             PCM = fixed("Rasch", 1, -cumsum(b)))

# Scores 0 to 3, expected score and information at theta = 1
th <- matrix(1)
t(sapply(mods, function(m) {
  it <- extract.item(m, 1)
  out <- c(probtrace(it, th), expected.item(it, th),
           iteminfo(it, th))
  setNames(round(out, 3), c(paste0("P", 0:3), "E", "info"))
}))

Uses mirt. Install once with install.packages(c("mirt")).

What the thresholds tell you