Concept map / Confidence intervals

Share

Confidence intervals and degrees of freedom

Explore the degrees of freedom of a sample mean and estimate interval coverage over repeated normal samples.

1How often does a 95% interval catch the mean?

Ten students average 51.3 on a test, with a 95% interval from 45.7 to 56.9. The population mean is fixed, so it is either inside that range or not. The 95% describes the method: build a confidence interval this way from many samples and about 95% of them catch the population mean, if the assumptions hold.

Each interval is the sample mean plus or minus a multiplier times the standard error from the central limit theorem. The standard deviation behind that error is estimated from the sample, so the multiplier comes from Student’s t with n − 1 degrees of freedom. Here every sample comes from a normal population with mean 50 and standard deviation 10.

Samples, newest lastTrue mean 50405060ScoreDraw samples to see their intervals
Captured the meanMissed the mean

Each 95% interval will use the Student’s t multiplier 2.26 on a sample of 10. Draw samples to see how often the intervals catch 50.

More controls

Changing a setting starts a new run.

2Samples of 5 with the normal multiplier

Samples of 5 come from a normal population. Each 95% interval is built with the normal multiplier 1.96 instead of Student’s t. Over many samples, how often do these intervals catch the population mean?

3Estimate plus or minus a multiple of the standard error

An interval for a mean is the sample mean plus or minus a multiplier times the standard error s/√n. Because s comes from the same sample, the multiplier t* comes from Student’s t with n − 1 degrees of freedom. Point at a symbol to light up its part of the chart or the multiplier buttons.

xˉ  ±  tn−1∗×sn\htmlClass{term-mean}{\bar x}\;\pm\;\htmlClass{term-tstar}{t^*_{n-1}}\times\htmlClass{term-se}{\frac{s}{\sqrt{n}}}

Draw a sample to fill this line with its numbers. With n = 10, the 95% multiplier is t* = 2.26, against 1.960 from the normal.

4Why n − 1: five values with a fixed mean

Fix the mean of five values at 10 and the total is 50. Choose any four values and the fifth is forced. Only four can vary freely, so the sample standard deviation divides by n − 1 = 4, and t* uses 4 degrees of freedom.

Value 510Set by the mean
x5=50−(8+9+11+12)=105−1=4 degrees of freedom\begin{gathered} x_5 = 50 - (8+9+11+12) = 10 \\[2pt] 5-1=4\text{ degrees of freedom} \end{gathered}

The first four values sum to 40, so the fifth must be 10 to keep the mean at 10.

5Check yourself

The tasks use the lab in section 1 and the five values in section 4, and tick themselves when you meet them.

  • In section 4, move the first four values so that the fifth has to be 14.Now: The first four sum to 40, so the fifth is 10
  • With samples of 10, 95% confidence and the t multiplier, draw at least 200 intervals. Coverage should land within 2 points of 95%.Now: n = 10, 95%, Student’s t multiplier: no intervals yet
  • Switch to the normal multiplier with samples of 5 and 95% confidence. Draw at least 100 intervals and watch coverage drop below 92%.Now: n = 10, 95%, Student’s t multiplier: no intervals yet

6A t interval and its coverage in R

t.test returns the interval for one sample, and the next lines rebuild it from the mean, qt and the standard error. The two replicate blocks repeat the lab’s coverage experiment with 10,000 samples each. The t intervals catch the mean in 0.9518 of them. With 1.96 on samples of 5, coverage falls to 0.8794, close to the exact 0.878 from pt.

A t interval and its coverageShow the code
# One sample of 10 from a normal population, mean 50, SD 10
set.seed(1)
x <- rnorm(10, mean = 50, sd = 10)
t.test(x, conf.level = 0.95)$conf.int

# The same interval by hand, with t on n - 1 = 9 df
se <- sd(x) / sqrt(10)
mean(x) + c(-1, 1) * qt(0.975, df = 9) * se

# Coverage over 10,000 samples of 10
hit <- function(ci) ci[1] <= 50 && 50 <= ci[2]
cover <- replicate(10000, {
  hit(t.test(rnorm(10, 50, 10))$conf.int)
})
mean(cover)

# Samples of 5 with 1.96 in place of the t multiplier
cover_z <- replicate(10000, {
  y <- rnorm(5, 50, 10)
  hit(mean(y) + c(-1, 1) * 1.96 * sd(y) / sqrt(5))
})
mean(cover_z)
# The exact coverage of that shortcut
1 - 2 * pt(-1.96, df = 4)

Uses base R only.

What the 95% means