Section 19.1

Estimators

Maximum likelihood, unbiased estimators (German tank, capture-recapture), posterior means under conjugate priors.

Toolkit

Maximum likelihood estimates parameters by maximising $\mathbb{P}(\text{data}\mid\theta)$ over $\theta$ ; for exponential-family models this reduces to matching sufficient statistics. Unbiasedness and conjugate Bayesian updating give alternative point estimators with known exact moments.

MLE: solve $\frac{d}{d\theta}\log L(\theta)=0$ ; for $X\sim\mathrm{Bin}(n,p)$ , $\hat p=k/n$ .
Method of moments / ratio estimators: equate observed proportions to population proportions, e.g. capture-recapture $\hat N = \frac{(\text{tagged})(\text{second catch})}{\text{recaptured}}$ .
Monte Carlo average of iid Bernoulli( $p$ ) indicators: $\hat\theta=c\bar X$ , $\operatorname{Var}(\hat\theta)=\frac{c^2 p(1-p)}{N}$ .
Beta–Binomial conjugacy: prior $\mathrm{Beta}(\alpha,\beta)$ , data $k$ successes in $n$ trials $\Rightarrow$ posterior $\mathrm{Beta}(\alpha+k,\beta+n-k)$ , posterior mean $\frac{\alpha+k}{\alpha+\beta+n}$ .
German tank / discrete uniform max: for $\mathrm{Unif}\{1,\dots,m\}$ sampled without replacement, the unbiased estimator of $m$ given maximum $k$ and sample size $n$ is $k\left(1+\frac1n\right)-1$ .

Problems