Concept map / Bayes
Share
Bayesian updating with coin tosses
Combine a beta prior with a binomial likelihood and compare how the posterior changes as observations accumulate.
1What ten tosses do to a prior belief
Before you toss a coin you may already have a view about p, its chance of heads. A prior distribution puts that view on the scale from 0 to 1. Here it has a mode m, the value it peaks at, and a strength k, the number of tosses it is worth.
The likelihood says how probable the observed tosses are at each value of p. Multiply it by the prior, rescale so the area is 1, and you have the posterior distribution. The lab starts with 7 heads in 10 tosses.
The prior peaks at 0.50 and counts as 20 tosses. 7 heads in 10 tosses give a proportion of 0.70. The posterior mode is 0.57, and the 95% credible interval runs from 0.39 to 0.73.
More controls
2Two priors, the same 100 tosses
Two people hold priors of the same strength, 20 tosses, but one peaks at 0.50 (green) and the other at 0.20 (blue). Both then see 70 heads in 100 tosses. How far apart are their posterior modes?
3The prior counts as k extra tosses
A prior of strength k at mode m acts like k earlier tosses with a share m of heads. Pool them with the n real tosses and the posterior mode is the share of heads in the combined pile.
(20 × 0.50 + 7) / (20 + 10) = 0.57. The prior carries 67% of the weight and the 10 tosses carry 33%. Point at a symbol to light up its curve.
4Check yourself
The tasks use the lab in section 1 and tick themselves when you meet them.
- Set the prior strength k to 0, which makes the prior uniform. The posterior mode should then equal the observed proportion of heads.Now: Strength 20, posterior mode 0.57, observed proportion 0.70
- Set the prior mode to 0.50 and the strength to 50 or more, then toss until the posterior mode passes 0.60.Now: Strength 20, posterior mode 0.57 after 10 flips
- Toss until the 95% credible interval is narrower than 0.20.Now: Range 0.39 to 0.73 (width 0.34)
5The same update in base R
The update is two additions, so base R is enough. The last lines repeat it by brute force on a grid, the way the chart is drawn, and both give a posterior mode of 0.5667.
Update a beta prior with coin tossesShow the code
# Beta prior with mode 0.5 and strength 20, as on the page
mode <- 0.5
strength <- 20
a <- 1 + mode * strength
b <- 1 + (1 - mode) * strength
# 7 heads in 10 tosses: add heads to a and tails to b
heads <- 7
tosses <- 10
a_post <- a + heads
b_post <- b + (tosses - heads)
post_mode <- (a_post - 1) / (a_post + b_post - 2)
ci <- qbeta(c(0.025, 0.975), a_post, b_post)
round(c(mode = post_mode, lower = ci[1], upper = ci[2]), 3)
# Same posterior by brute force on a grid
p <- seq(0.0001, 0.9999, by = 0.0001)
post <- dbeta(p, a, b) * dbinom(heads, tosses, p)
post <- post / sum(post)
p[which.max(post)] # 0.5667, the same as post_modeUses base R only.
What the posterior does
- With a beta prior and coin tosses, the heads are added to the prior’s first shape parameter and the tails to the second.
- The posterior mode is (k × m + h) / (k + n). A prior of strength 20 at 0.50 and 7 heads in 10 tosses give 0.57, and once n is well past k the observed proportion dominates.
- A flat prior, k = 0, leaves the posterior mode at the observed proportion.
- A 95% credible interval holds 95% of the posterior probability. With enough tosses it narrows, whatever the prior.
- In item response theory, a prior on ability keeps the estimate finite for someone who gets every item right or every item wrong.
Recommended reading
- Statistical RethinkingRichard McElreath2nd ed., chapters 2–3, Bayes and working with posterior draws