Rao–Blackwell theorem

The Rao–Blackwell theorem states that conditioning an unbiased estimator on a sufficient statistic preserves its expectation and cannot increase its variance, when the required moments exist.

Sufficient statistic: A statistic that preserves all information in a sample about an unknown parameter under a specified model. The theorem conditions on this statistic to remove variation unrelated to the parameter.
Bernoulli distribution: A probability distribution on two outcomes, conventionally coded as zero and one. For Bernoulli samples, the success count is sufficient for the success probability.
Statistical estimator: A rule that maps observed data to an estimate of an unknown quantity. The theorem transforms one estimator into another using sample information.
Cramér–Rao bound: A lower bound on the variance of unbiased estimators under regularity conditions. It gives a bound, while Rao–Blackwell constructs a no-worse estimator from an existing one.
Conditional expectation: The best mean-square prediction of a random variable given specified information. It defines the estimator produced by the Rao–Blackwell procedure.
Binomial distribution: The distribution of the number of successes in a fixed number of independent trials with common success probability. Its sufficient success count can replace individual outcomes when conditioning an estimator.
Variance: A measure of the average squared deviation of a random variable from its mean. The theorem compares the variances of the original and conditioned estimators.
Biased estimator: An estimator whose expected value differs from the quantity being estimated for at least one parameter value. The theorem’s stated preservation guarantee concerns unbiasedness, not arbitrary bias–variance tradeoffs.
Law of total variance: An identity decomposing variance into expected conditional variance and variance of the conditional mean. It explains why conditioning cannot increase variance.
Sample mean: The arithmetic average of observations in a sample. In common models, conditioning a sample-based estimate on a sufficient statistic yields a function of the sample mean.

Linked from 4 pages

Basu's theoremCompared with: It uses sufficiency to improve estimators rather than to prove independence.