Ridge regression

Ridge regression estimates regression coefficients by minimizing residual sum of squares plus a penalty proportional to their squared magnitudes. The penalty shrinks coefficients toward zero without usually making them exactly zero.

Tikhonov regularization: A method for stabilizing ill-posed inverse problems by penalizing the size or structure of a solution. Ridge regression is its squared-norm form for linear least squares.
Linear regression: A method that models an outcome as a linear combination of predictors and estimates coefficients from data. Ridge modifies this model's least-squares fitting objective.
Lasso (statistics): A regression method that adds an absolute-value coefficient penalty to least squares. Unlike ridge, its penalty can set some coefficients exactly to zero.
Genomics: The study of the structure, function, evolution, and mapping of genomes. Many genomic prediction problems have far more correlated predictors than observations.
L2 norm: The square root of the sum of squared vector components, measuring Euclidean length. Ridge penalizes the squared L2 norm of the coefficient vector.
Ordinary least squares: The coefficient estimator that minimizes the sum of squared residuals without a coefficient penalty. It is the unregularized objective that ridge regression extends.
Elastic net regularization: A regression penalty combining L1 and L2 terms. It combines ridge-like shrinkage with lasso's capacity for sparse coefficients.
Kernel ridge regression: A kernel-based regression method that combines least-squares fitting with a squared-norm penalty. It extends ridge regularization to nonlinear relationships through kernel functions.
Bias–variance tradeoff: The balance between systematic prediction error from simplifying a model and error from sensitivity to sampled data. Increasing the penalty raises bias while often reducing variance.
Multicollinearity: A condition in which predictors in a regression model are strongly linearly related. Correlated predictors can make ordinary least-squares coefficients unstable, motivating ridge shrinkage.

Linked from 8 pages