Parameter estimation

Parameter estimation uses observed data to infer values for quantities that define a statistical or mathematical model. The resulting estimates depend on the model, data, and estimation method.

Bayesian inference: A framework that updates probability distributions for unknown quantities using evidence and prior information. It combines observed data with prior distributions to produce parameter estimates and uncertainty.
Statistical model: A mathematical description of how data arise, including systematic structure and random variation. Parameter estimation presupposes a model connecting unknown quantities to the observations.
Linear regression: A statistical method modeling an outcome as a linear combination of predictors and an error term. Its coefficients are commonly estimated from paired predictor and outcome data.
Point estimation: The use of sample data to produce a single numerical estimate of an unknown quantity. It is the specific form of estimation that reports one parameter value.
Bias–variance tradeoff: The tradeoff between systematic estimation error and variability across samples in predictive models. Estimation choices balance systematic error against sensitivity to the particular sample.
Least squares: An estimation method that minimizes the sum of squared differences between observed and predicted values. It estimates model parameters by minimizing residual error in regression and related models.
Identifiability: A property describing whether distinct parameter values imply distinguishable probability distributions or observations. If parameters are not identifiable, data cannot uniquely determine their values.
Nonlinear regression: Regression in which the predicted response depends nonlinearly on model parameters. Its parameters are inferred from data using nonlinear optimization.
Interval estimation: The construction of a range of plausible values for an unknown quantity using sample data. Unlike a point estimate, it represents uncertainty with a range.
Maximum likelihood estimation: A method that chooses parameter values maximizing the probability of observed data under a model. Its consistency and finite-sample behavior depend on model assumptions and data conditions.

Linked from 15 pages