Regularization

Regularization stabilizes an ill-posed problem by adding constraints or modifying its objective. It favors solutions with desired properties, often at the cost of some fit to observed data.

Tikhonov regularization: A method that stabilizes an inverse problem by penalizing the size of its solution, often through a quadratic penalty. Its penalty term gives a canonical way to suppress unstable solutions.
Inverse problem: A problem that infers hidden causes or parameters from observed effects. Regularization is commonly needed when recovering causes from incomplete or noisy measurements.
L0 regularization: A penalty based on the number of nonzero entries in a parameter vector. It directly counts selected variables, unlike convex sparsity-promoting penalties.
Jacques Hadamard: A French mathematician whose work helped establish the modern framework for well-posed problems. His criteria clarify the instability that regularization is designed to address.
Lasso (statistics): A regression method that uses an L1 penalty to encourage sparse coefficients. Its absolute-value penalty can eliminate coefficients while regularizing the fit.
Ill-posed problem: A problem that lacks a solution, has nonunique solutions, or changes unstably with its data. Regularization addresses the instability or nonuniqueness that makes these problems unreliable.
Elastic net regularization: A regression penalty combining L1 and squared L2 penalties. It combines sparse selection with the coefficient grouping encouraged by ridge-like shrinkage.
Andrey Tikhonov: A Soviet mathematician known for developing a method of stabilizing ill-posed inverse problems. His name is attached to a foundational penalty-based regularization method.
Constrained optimization: Optimization in which candidate solutions must satisfy specified equality or inequality conditions. A feasible-set restriction is an alternative way to impose regularizing structure.
Image deconvolution: The reconstruction of an image from data blurred by a known or estimated imaging process. Regularization prevents noise from dominating the recovered image.

Linked from 19 pages