Concept map / Bayes

Share

Bayesian updating with coin tosses

Combine a beta prior with a binomial likelihood and compare how the posterior changes as observations accumulate.

Prerequisites Probability, LikelihoodNext topics MCMC, Ability estimation, DCM

1What ten tosses do to a prior belief

Before you toss a coin you may already have a view about p, its chance of heads. A prior distribution puts that view on the scale from 0 to 1. Here it has a mode m, the value it peaks at, and a strength k, the number of tosses it is worth.

The likelihood says how probable the observed tosses are at each value of p. Multiply it by the prior, rescale so the area is 1, and you have the posterior distribution. The lab starts with 7 heads in 10 tosses.

00.20.40.60.81Relative densityCoin bias p (chance of heads)
Prior densityNormalized likelihoodPosterior density95% equal-tail credible interval

The prior peaks at 0.50 and counts as 20 tosses. 7 heads in 10 tosses give a proportion of 0.70. The posterior mode is 0.57, and the 95% credible interval runs from 0.39 to 0.73.

More controls

2Two priors, the same 100 tosses

Two people hold priors of the same strength, 20 tosses, but one peaks at 0.50 (green) and the other at 0.20 (blue). Both then see 70 heads in 100 tosses. How far apart are their posterior modes?

00.20.40.60.81DensityCoin bias pafter 0 flips: 0.30 apart

3The prior counts as k extra tosses

A prior of strength k at mode m acts like k earlier tosses with a share m of heads. Pool them with the n real tosses and the posterior mode is the share of heads in the combined pile.

posterior mode  =  k×m  +  hk+n\text{posterior mode} \;=\; \frac{\htmlClass{term-prior}{k \times m} \;+\; \htmlClass{term-data}{h}}{\htmlClass{term-prior}{k} + \htmlClass{term-data}{n}}

(20 × 0.50 + 7) / (20 + 10) = 0.57. The prior carries 67% of the weight and the 10 tosses carry 33%. Point at a symbol to light up its curve.

4Check yourself

The tasks use the lab in section 1 and tick themselves when you meet them.

  • Set the prior strength k to 0, which makes the prior uniform. The posterior mode should then equal the observed proportion of heads.Now: Strength 20, posterior mode 0.57, observed proportion 0.70
  • Set the prior mode to 0.50 and the strength to 50 or more, then toss until the posterior mode passes 0.60.Now: Strength 20, posterior mode 0.57 after 10 flips
  • Toss until the 95% credible interval is narrower than 0.20.Now: Range 0.39 to 0.73 (width 0.34)

5The same update in base R

The update is two additions, so base R is enough. The last lines repeat it by brute force on a grid, the way the chart is drawn, and both give a posterior mode of 0.5667.

Update a beta prior with coin tossesShow the code
# Beta prior with mode 0.5 and strength 20, as on the page
mode <- 0.5
strength <- 20
a <- 1 + mode * strength
b <- 1 + (1 - mode) * strength

# 7 heads in 10 tosses: add heads to a and tails to b
heads <- 7
tosses <- 10
a_post <- a + heads
b_post <- b + (tosses - heads)

post_mode <- (a_post - 1) / (a_post + b_post - 2)
ci <- qbeta(c(0.025, 0.975), a_post, b_post)
round(c(mode = post_mode, lower = ci[1], upper = ci[2]), 3)

# Same posterior by brute force on a grid
p <- seq(0.0001, 0.9999, by = 0.0001)
post <- dbeta(p, a, b) * dbinom(heads, tosses, p)
post <- post / sum(post)
p[which.max(post)]  # 0.5667, the same as post_mode

Uses base R only.

What the posterior does