Concept map / Bernoulli distribution
Share
The Bernoulli distribution
Model a single correct or incorrect response, compare simulated responses with the Bernoulli probabilities, and see why an item's score variance peaks at p = 0.5.
1One item, two possible scores
Score a test item 1 for a right answer and 0 for a wrong one. If each test taker answers correctly with probability p, the score is a Bernoulli random variable: 1 with probability p and 0 with probability 1 − p. That single number describes the whole distribution, and the rules of probability are all it needs.
How much do scores on one item vary, and does an easy item vary more than a hard one? Set p, draw simulated responses, and see how close the observed proportion correct comes to p.
With p = 0.60, a test taker scores 1 with probability 60.0% and 0 with probability 40.0%. The mean score is 0.60 and the variance is 0.240.
2An easy item and a hard item: which varies more?
90% of test takers get item A right and 10% get item B right. Each response is scored 1 or 0. Which item's scores have the larger variance?
3Why the variance peaks at p = 0.5
Averaged over many test takers, a score of 1 or 0 comes to p. The variance is the product of the two chances, so it is small whenever either chance is small. Point at p or 1 − p and the matching bar and point light up.
Now: Var(X) = 0.60 × 0.40 = 0.240, a standard deviation of 0.490.
Filled: the current p. Open: 1 − p = 0.40, with the same variance.
4Check yourself
The tasks use the slider and buttons in section 1 and tick themselves when you meet them.
- Set p so that the variance of the score is as large as it can be.Now: p = 0.60, variance 0.240
- At p = 0.20 the variance is 0.160. Find the other value of p that gives the same variance.Now: p = 0.60, variance 0.240
- Draw at least 200 responses with p strictly between 0 and 1, and get the observed proportion within 0.02 of p.Now: no responses drawn yet
5Simulate item scores in R
R has no Bernoulli function of its own. A binomial with size = 1 is one Bernoulli trial. The snippet draws 200 responses at p = 0.6, sets the observed proportion beside p, and prints p(1 − p) at four values of p. The last lines take p from a Rasch item.
Simulate right-or-wrong responsesShow the code
# One item answered correctly with probability p = 0.6
set.seed(2024)
p <- 0.6
# A Bernoulli draw is a binomial draw with size = 1
scores <- rbinom(200, size = 1, prob = p)
table(scores)
# Observed proportion against the model mean p
round(c(observed = mean(scores), model = p), 3)
# Model variance p(1 - p), the same at p and at 1 - p
p_grid <- c(0.1, 0.5, 0.6, 0.9)
round(p_grid * (1 - p_grid), 3)
# Item response theory: P(correct) from a Rasch item
theta <- 0.5
b <- 0
p_irt <- plogis(theta - b)
round(c(p = p_irt, variance = p_irt * (1 - p_irt)), 3)Uses base R only.
What one item score holds
- The expected score is p, so the proportion correct in a large sample estimates p.
- The variance is p(1 − p). It peaks at 0.25 when p = 0.5 and drops to 0 when the answer is certain.
- An item that 90% of test takers get right has the same variance, 0.09, as one that 10% get right. Swapping p and 1 − p leaves the product unchanged.
- Add up independent Bernoulli scores with the same p and you get a binomial count. Count the trials up to the first success and you get a geometric count.
- In item response theory each response is still a Bernoulli trial, but its p depends on the test taker’s ability and the item’s difficulty, so it changes from person to person and item to item.
Recommended reading
- Introduction to Probability, Statistics, and Random ProcessesHossein Pishro-Nik§3.1.5, Bernoulli random variables