Concept map / Bernoulli distribution

Share

The Bernoulli distribution

Model a single correct or incorrect response, compare simulated responses with the Bernoulli probabilities, and see why an item's score variance peaks at p = 0.5.

1One item, two possible scores

Score a test item 1 for a right answer and 0 for a wrong one. If each test taker answers correctly with probability p, the score is a Bernoulli random variable: 1 with probability p and 0 with probability 1 − p. That single number describes the whole distribution, and the rules of probability are all it needs.

How much do scores on one item vary, and does an easy item vary more than a hard one? Set p, draw simulated responses, and see how close the observed proportion correct comes to p.

0.000.250.500.751.00ProbabilityScore on the item0.400, incorrect0.601, correct
P(score 1) = pP(score 0) = 1 − p

With p = 0.60, a test taker scores 1 with probability 60.0% and 0 with probability 40.0%. The mean score is 0.60 and the variance is 0.240.

2An easy item and a hard item: which varies more?

90% of test takers get item A right and 10% get item B right. Each response is scored 1 or 0. Which item's scores have the larger variance?

3Why the variance peaks at p = 0.5

Averaged over many test takers, a score of 1 or 0 comes to p. The variance is the product of the two chances, so it is small whenever either chance is small. Point at p or 1 − p and the matching bar and point light up.

E[X]=pVar⁡(X)=p (1−p)\begin{gathered} E[X] = \htmlClass{term-one}{p} \\ \operatorname{Var}(X) = \htmlClass{term-one}{p}\,\htmlClass{term-zero}{(1-p)} \end{gathered}

Now: Var(X) = 0.60 × 0.40 = 0.240, a standard deviation of 0.490.

0.10.20.30.00.20.40.60.81.0Variance of the scoreProbability of a correct response, p

Filled: the current p. Open: 1 − p = 0.40, with the same variance.

4Check yourself

The tasks use the slider and buttons in section 1 and tick themselves when you meet them.

  • Set p so that the variance of the score is as large as it can be.Now: p = 0.60, variance 0.240
  • At p = 0.20 the variance is 0.160. Find the other value of p that gives the same variance.Now: p = 0.60, variance 0.240
  • Draw at least 200 responses with p strictly between 0 and 1, and get the observed proportion within 0.02 of p.Now: no responses drawn yet

5Simulate item scores in R

R has no Bernoulli function of its own. A binomial with size = 1 is one Bernoulli trial. The snippet draws 200 responses at p = 0.6, sets the observed proportion beside p, and prints p(1 − p) at four values of p. The last lines take p from a Rasch item.

Simulate right-or-wrong responsesShow the code
# One item answered correctly with probability p = 0.6
set.seed(2024)
p <- 0.6

# A Bernoulli draw is a binomial draw with size = 1
scores <- rbinom(200, size = 1, prob = p)
table(scores)

# Observed proportion against the model mean p
round(c(observed = mean(scores), model = p), 3)

# Model variance p(1 - p), the same at p and at 1 - p
p_grid <- c(0.1, 0.5, 0.6, 0.9)
round(p_grid * (1 - p_grid), 3)

# Item response theory: P(correct) from a Rasch item
theta <- 0.5
b <- 0
p_irt <- plogis(theta - b)
round(c(p = p_irt, variance = p_irt * (1 - p_irt)), 3)

Uses base R only.

What one item score holds