Section 21.1

Keep, Switch or Reroll Once

A single strategic choice: keep or reroll once, stay or switch urns, choose a threshold or allocation, guess optimally.

Toolkit

These problems all reduce to a single compare-and-commit decision: compute the expected value of each available action, conditioning on whatever information has already been revealed, and pick the action with the larger (or smaller, for costs) expected value. Key tools:

Reroll/keep rule: keep a current realisation $x$ iff $x \ge \mathbb{E}[\text{fresh draw}]$ (adjusting the fresh draw’s expectation for any penalties or conditioning).
Bayesian updating: after observing data, update $\mathbb{P}(\text{state})$ via
$\mathbb{P}(H\mid D) = \frac{\mathbb{P}(D\mid H)\mathbb{P}(H)}{\sum_{H'}\mathbb{P}(D\mid H')\mathbb{P}(H')},$
then compare the resulting conditional expected values of each option.
Optimising a parametric choice (threshold, number, allocation): write expected payoff as a function of the parameter, differentiate or complete the square for concave quadratics, and solve for the vertex, respecting boundary/integrality constraints.
Linearity of expectation lets per-step optimal decisions be summed independently when the choices do not interact.

Problems