Concept map / Confidence intervals
Share
Confidence intervals and degrees of freedom
Explore the degrees of freedom of a sample mean and estimate interval coverage over repeated normal samples.
1How often does a 95% interval catch the mean?
Ten students average 51.3 on a test, with a 95% interval from 45.7 to 56.9. The population mean is fixed, so it is either inside that range or not. The 95% describes the method: build a confidence interval this way from many samples and about 95% of them catch the population mean, if the assumptions hold.
Each interval is the sample mean plus or minus a multiplier times the standard error from the central limit theorem. The standard deviation behind that error is estimated from the sample, so the multiplier comes from Student’s t with n − 1 degrees of freedom. Here every sample comes from a normal population with mean 50 and standard deviation 10.
Each 95% interval will use the Student’s t multiplier 2.26 on a sample of 10. Draw samples to see how often the intervals catch 50.
More controls
Changing a setting starts a new run.
2Samples of 5 with the normal multiplier
Samples of 5 come from a normal population. Each 95% interval is built with the normal multiplier 1.96 instead of Student’s t. Over many samples, how often do these intervals catch the population mean?
3Estimate plus or minus a multiple of the standard error
An interval for a mean is the sample mean plus or minus a multiplier times the standard error s/√n. Because s comes from the same sample, the multiplier t* comes from Student’s t with n − 1 degrees of freedom. Point at a symbol to light up its part of the chart or the multiplier buttons.
Draw a sample to fill this line with its numbers. With n = 10, the 95% multiplier is t* = 2.26, against 1.960 from the normal.
4Why n − 1: five values with a fixed mean
Fix the mean of five values at 10 and the total is 50. Choose any four values and the fifth is forced. Only four can vary freely, so the sample standard deviation divides by n − 1 = 4, and t* uses 4 degrees of freedom.
The first four values sum to 40, so the fifth must be 10 to keep the mean at 10.
5Check yourself
The tasks use the lab in section 1 and the five values in section 4, and tick themselves when you meet them.
- In section 4, move the first four values so that the fifth has to be 14.Now: The first four sum to 40, so the fifth is 10
- With samples of 10, 95% confidence and the t multiplier, draw at least 200 intervals. Coverage should land within 2 points of 95%.Now: n = 10, 95%, Student’s t multiplier: no intervals yet
- Switch to the normal multiplier with samples of 5 and 95% confidence. Draw at least 100 intervals and watch coverage drop below 92%.Now: n = 10, 95%, Student’s t multiplier: no intervals yet
6A t interval and its coverage in R
t.test returns the interval for one sample, and the next lines rebuild it from the mean, qt and the standard error. The two replicate blocks repeat the lab’s coverage experiment with 10,000 samples each. The t intervals catch the mean in 0.9518 of them. With 1.96 on samples of 5, coverage falls to 0.8794, close to the exact 0.878 from pt.
A t interval and its coverageShow the code
# One sample of 10 from a normal population, mean 50, SD 10
set.seed(1)
x <- rnorm(10, mean = 50, sd = 10)
t.test(x, conf.level = 0.95)$conf.int
# The same interval by hand, with t on n - 1 = 9 df
se <- sd(x) / sqrt(10)
mean(x) + c(-1, 1) * qt(0.975, df = 9) * se
# Coverage over 10,000 samples of 10
hit <- function(ci) ci[1] <= 50 && 50 <= ci[2]
cover <- replicate(10000, {
hit(t.test(rnorm(10, 50, 10))$conf.int)
})
mean(cover)
# Samples of 5 with 1.96 in place of the t multiplier
cover_z <- replicate(10000, {
y <- rnorm(5, 50, 10)
hit(mean(y) + c(-1, 1) * 1.96 * sd(y) / sqrt(5))
})
mean(cover_z)
# The exact coverage of that shortcut
1 - 2 * pt(-1.96, df = 4)Uses base R only.
What the 95% means
- The confidence level belongs to the method. Any one interval either contains the mean or does not.
- Once the mean is fixed, only n − 1 values can vary, which is why the sample variance divides by n − 1 and t uses n − 1 degrees of freedom.
- With n = 10 the 95% multiplier is 2.26, not 1.96. With samples of 5, intervals built with 1.96 catch the mean 87.8% of the time.
- Width shrinks with √n, so halving it takes four times the data. A 99% interval with n = 10 uses 3.25 and is about 44% wider than the 95% one.
- The same plus-or-minus logic gives the band around an observed score in classical test theory, and hypothesis tests run the calculation the other way.
Recommended reading
- Introductory Statistics 2eBarbara Illowsky and Susan DeanChapter 8, confidence intervals for a mean and Student’s t