Fisher information

Fisher information measures how sensitively a probability distribution changes with an unknown parameter, indicating how much observations can reveal about that parameter.

Score (statistics): The derivative of a log-likelihood with respect to a model parameter. Fisher information is the expected squared score under regularity conditions.
Maximum likelihood estimation: A method that estimates parameters by maximizing the likelihood of observed data. Its estimator often approaches the inverse-information variance bound in large samples.
Ronald Fisher: A British statistician and geneticist who developed foundational methods in modern statistics. Fisher introduced the information measure in his work on estimation and likelihood.
Shannon entropy: A measure of the average uncertainty in a random variable's outcomes. Entropy measures outcome uncertainty, while Fisher information measures sensitivity to a parameter.
Likelihood function: A function measuring how well parameter values explain observed data. Fisher information quantifies the local curvature of likelihood around a parameter value.
Asymptotic normality: A property in which a statistic's sampling distribution approaches a normal distribution as sample size grows. Under regularity conditions, maximum-likelihood errors scale according to inverse Fisher information.
Statistical estimation: The use of sample data to infer unknown features of a probability model. Fisher information was developed to characterize the precision available in parameter estimation.
Mutual information: A measure of how much knowing one random variable reduces uncertainty about another. It measures dependence between variables rather than local sensitivity to a model parameter.
Cramér–Rao bound: A lower bound on the variance of unbiased estimators under specified regularity conditions. Fisher information determines the bound on achievable estimator precision.
Optimal experimental design: The selection of experimental conditions to improve parameter estimation. Design criteria often maximize or otherwise optimize the experiment's information matrix.

Linked from 11 pages