Section 5.3

Which Coin Was It?

A random source (coin, urn, shelf) is chosen and observed: its posterior, how many observations give confidence, and the predictive probability of the next observation.

Toolkit

Core tool: Bayes’ theorem applied to a discrete prior over which source (coin/urn/shelf) was selected, updated by i.i.d. observations from that source.

\mathbb{P}(\text{source}=i\mid \text{data}) = \frac{\mathbb{P}(\text{data}\mid i)\,\pi_i}{\sum_j \mathbb{P}(\text{data}\mid j)\,\pi_j}.

Work in odds form when comparing two sources: posterior odds = prior odds $\times$ likelihood ratio, and each independent observation multiplies the odds by that observation’s likelihood ratio.

For predictive (next-observation) probabilities, do not just plug in a point estimate: average the next-outcome probability over the full posterior,

\mathbb{P}(\text{next}=\text{success}\mid \text{data}) = \sum_i \mathbb{P}(\text{next success}\mid i)\,\mathbb{P}(i\mid\text{data}),

which for a Beta-like mixture of success probabilities $p$ reduces to $\mathbb{E}[p^{k+1}]/\mathbb{E}[p^k]$ after $k$ successes in a row. Confidence grows as the likelihood ratio compounds geometrically in the number of consistent observations.

Problems