Maximum likelihood estimation

Maximum likelihood estimation chooses model parameter values that maximize the probability of the observed data under the model.

Likelihood function: A function of model parameters that assigns each parameter setting the probability or density of the observed data. It is the objective function that maximum likelihood estimation seeks to maximize.
Statistical model: A mathematical description of possible data distributions, often indexed by unknown parameters. Maximum likelihood can estimate parameters only relative to a specified model.
Logistic regression: A regression model that represents event probabilities through a logistic function of predictors. Its coefficients are commonly estimated by maximizing the likelihood of observed outcomes.
Bayesian estimation: Inference that combines a likelihood with a prior distribution to obtain a posterior distribution over parameters. Unlike maximum likelihood, it incorporates prior information and yields a parameter distribution.
Consistency (statistics): A property of an estimator that converges in probability to the true parameter as sample size grows. Under suitable conditions, maximum likelihood estimates converge to the generating parameter.
Log-likelihood: The logarithm of a likelihood function, often used to simplify optimization without changing its maximizer. Taking logarithms turns products of independent observations into sums while preserving the maximum.
Probability distribution: A mathematical rule assigning probabilities or probability densities to possible outcomes. The model's distribution determines the likelihood assigned to the observed data.
Poisson regression: A regression model for count outcomes whose conditional mean depends on predictors. Its parameters are often fitted by maximizing a Poisson likelihood.
Method of moments: An estimation method that matches sample moments to moments implied by a model. It can produce estimates without maximizing the data likelihood.
Asymptotic normality: The convergence of a suitably scaled estimator's sampling distribution toward a normal distribution as sample size grows. Maximum likelihood estimates are often approximately normal in large samples.

Linked from 55 pages