Concept map / Logistic regression
Share
Logistic regression for pass and fail
Fit an S-shaped curve to pass/fail outcomes, read the slope as a change in log odds, and see that an item curve is the same model.
1Forty students, pass or fail
Forty students report how many hours they studied, and each one passed the exam or failed it. A regression line is the obvious tool, but the outcome is 0 or 1, and what you want to predict is a chance, which has to stay between 0 and 1. Logistic regression fits its straight line to the log odds of passing instead, and on the probability scale that line becomes an S-shaped curve.
Move the intercept β₀ and the slope β₁ to fit the curve by eye. The log-likelihood, the likelihood of all 40 outcomes on a log scale, rises as the curve gets better, and the fit button finds the curve where it peaks. The x slider reads off the chance of passing at any number of hours.
At 5.0 hours the log odds are −2.0 + 0.30 × 5.0 = −0.50, so the odds are 0.61 to 1 and the chance of passing is 38%. Each extra hour multiplies the odds by 1.35. The log-likelihood is −22.11; the best curve reaches −17.01.
2One more hour, at 2 hours or at 5
On the maximum likelihood curve, each extra hour multiplies the odds of passing by 2.25. Student A has studied 2 hours and has a 0.09 chance of passing. Student B has studied 5 hours and has 0.52. Each studies one hour more. Whose chance of passing goes up by more?
3Slopes live on the log-odds scale
The model is a straight line, but for the log odds of passing, not for the chance itself. Odds are p / (1 − p): a 75% chance is odds of 3 to 1. Log odds run from minus to plus infinity, so a line can never leave their range, and the logistic function bends that line back into an S between 0 and 1.
At x = 5.0: log odds = −2.0 + 0.30 × 5.0 = −0.50; odds = e−0.50 = 0.61; p = 38%. Point at a symbol to find it on the chart in section 1.
e0.30 = 1.35. This ratio is the same at every x, which is why β₁ is reported as an odds ratio. The change in probability is not the same: it is largest near 50%, where the curve is steepest.
How the fit button finds β₀ and β₁
Each passing student contributes log pi and each failing student log(1 − pi). The sum is the log-likelihood:
No formula gives the maximum directly. The button climbs to it with Newton–Raphson steps, 7 of them here, the same method glm() uses under the name iteratively reweighted least squares. The result is β₀ = −3.971 and β₁ = 0.811, with standard errors 1.29 and 0.24.
4Why a straight line will not do
Least squares will happily fit a line to 0s and 1s. On these students it rises 0.133 per hour everywhere, and it promises a chance of −0.16 at 0 hours and 1.17 at 10. Neither is a probability. The flat slope also misses the shape of the data: almost nobody passes below 3 hours and almost everybody passes above 7.
5An item curve is this same model
Replace hours with ability θ and pass with a correct answer, and the logistic curve becomes the two-parameter item characteristic curve. Only the way it is written changes. The slope is the discrimination a, and the difficulty b is the point where the curve crosses 50%.
Read as an item, the curve in section 1 has a = 0.30 and b = −(−2.0) / 0.30 = 6.67 hours. mirt reports the intercept itself as d, so its d is β₀.
The one real difference is that ability is never observed. Here hours are measured and the fit is an ordinary regression; in IRT, θ has to be estimated together with a and b, which is what calibration does.
6Check yourself
- Without the fit button, move β₀ and β₁ until the log-likelihood is within 0.2 of its maximum, −17.01.Now: log-likelihood −22.11, which is 5.09 below the maximum
- Fit the curve by maximum likelihood, then find the hours of study that give an even chance of passing.Now: your own curve is showing, not the fitted one
- On your own curve, build an item with discrimination a = 1.5 and difficulty b = 4 hours.Now: a = 0.30, b = 6.67 hours
7Fit the same model with glm()
The snippet types in the lab’s 40 students and fits them with glm(family = binomial). It prints β₀ = −3.971 and β₁ = 0.811, a log-likelihood of −17.01, an odds ratio of 2.25 and a 50% point of 4.90 hours, the numbers the fit button gives. The chances at 2, 3, 5 and 6 hours are the ones in section 2, and the last line is the straight line from section 4.
Fit a logistic regression with glm()Show the code
# The lab's 40 students: hours of study, pass (1) or fail (0)
hours <- c(1, 1.4, 1.5, 1.5, 1.9, 2.3, 2.3, 2.4, 3, 3.1, 3.3,
3.5, 3.8, 4.2, 4.4, 4.5, 4.6, 4.7, 5, 5.4, 5.4, 5.8, 6.1,
6.2, 6.5, 6.6, 6.7, 6.9, 7, 7, 7.1, 7.4, 7.9, 8.2, 8.6, 8.7,
8.9, 9, 9.3, 9.7)
pass <- c(0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 1, 0, 0,
1, 1, 0, 0, 1, 1, 0, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1)
# Maximum likelihood fit of log odds = b0 + b1 * hours
fit <- glm(pass ~ hours, family = binomial)
round(coef(fit), 3)
logLik(fit)
# Odds ratio for one more hour, and the 50% point -b0 / b1
exp(coef(fit)[["hours"]])
-coef(fit)[[1]] / coef(fit)[[2]]
# Chance of passing at 2, 3, 5 and 6 hours
new <- data.frame(hours = c(2, 3, 5, 6))
round(predict(fit, new, type = "response"), 3)
# A straight line predicts below 0 and above 1
line <- lm(pass ~ hours)
round(predict(line, data.frame(hours = c(0, 10))), 2)Uses base R only.
What the curve tells you
- Logistic regression fits a straight line to the log odds of the outcome, log(p / (1 − p)), and the logistic function turns that line into an S-shaped curve that stays between 0 and 1.
- β₁ is the change in log odds for one more hour. Its exponential is the odds ratio, 2.25 here, and it is the same at every number of hours.
- A change in probability depends on where you start. One more hour adds 0.09 at 2 hours and 0.19 at 5, and it never adds much more than β₁ / 4 = 0.20, the slope at the 50% point.
- An item characteristic curve is a logistic regression of the item response on ability, with a = β₁ and b = −β₀ / β₁. The item curves page starts from there.
Recommended reading
- Regression and Other StoriesAndrew Gelman, Jennifer Hill, and Aki VehtariChapters 13–14, logistic regression and working with it
- Detecting differential item functioning using logistic regression proceduresHariharan Swaminathan and H. Jane RogersPaper · Journal of Educational Measurement, 27(4), 1990