Sufficient statistic

A statistic is sufficient for a model parameter when, given its value, the sample contains no additional information about that parameter.

Neyman–Fisher factorization theorem: A theorem stating that a statistic is sufficient when the likelihood factors into a parameter-dependent term involving that statistic and a parameter-free term. It converts sufficiency into a practical factorization test on the likelihood.
Statistic: A function of observed data that does not depend on unknown model parameters. Sufficiency is a special property that a statistic can have.
Rao–Blackwell theorem: A theorem showing that conditioning an estimator on a sufficient statistic can reduce or preserve its variance. It turns a sufficient statistic into a tool for improving unbiased estimators.
Unbiased estimator: An estimator whose expected value equals the parameter it estimates for every parameter value. Unbiasedness concerns an estimator's expectation; sufficiency concerns information retained by a statistic.
Conditional distribution: The probability distribution of one random variable given the value of another. Sufficiency means the sample's conditional distribution given the statistic does not depend on the parameter.
Statistical sample: A collection of observed values treated as data generated by a probability model. The statistic summarizes this sample, whose parameter information it must preserve.
Maximum likelihood estimation: A method that estimates parameters by maximizing the likelihood of observed data. Likelihood factorization can show that maximum-likelihood calculations depend on the sample only through a sufficient statistic.
Fisher information: A measure of the information a probability model's data provide about an unknown parameter. It quantifies parameter sensitivity, while sufficiency asks whether a statistic preserves all sample information.
Likelihood function: A function of model parameters proportional to the probability or density of observed data. Its factorization exposes whether the data's parameter dependence passes entirely through the statistic.
Statistical parameter: A numerical quantity that specifies a feature of a probability model, such as its mean or variance. Sufficiency is always defined relative to a particular model parameter.

Linked from 11 pages