Multicollinearity

Multicollinearity is strong linear dependence among predictors in a regression model. It makes separate predictor effects difficult to estimate precisely, even when predictions remain accurate.

Variance inflation factor: A statistic measuring how much a regression coefficient’s variance increases because of predictor dependence. It quantifies how strongly each predictor’s coefficient uncertainty is inflated by the others.
Ridge regression: A regression method that penalizes large coefficients using the sum of their squares. Its penalty stabilizes estimates when predictors are strongly correlated.
Homoscedasticity: A condition in which a model’s errors have constant variance across predictor values. It concerns error variance, whereas multicollinearity concerns dependence among predictors.
Partial regression plot: A plot showing the relationship between an outcome and one predictor after removing other predictors’ linear effects. It visualizes a predictor’s unique contribution, which can be weak under strong dependence.
Condition number: A numerical measure of how sensitive a matrix-based calculation is to small changes in its inputs. A large design-matrix condition number signals near-dependencies that can make estimates unstable.
Principal component regression: A regression method that predicts outcomes using selected principal components of the predictors. It replaces dependent predictors with uncorrelated components, often reducing coefficient instability.
Endogeneity: A condition in which a predictor is correlated with a regression error term. Unlike multicollinearity, endogeneity can bias coefficient estimates even with abundant data.
Bootstrap (statistics): A resampling method that estimates uncertainty by repeatedly drawing samples from observed data. Bootstrap coefficient distributions can reveal instability that a single fitted model conceals.
Design matrix: A matrix whose rows represent observations and whose columns represent variables used in a statistical model. Multicollinearity is dependence among the predictor columns of this matrix.
Lasso: A regression method that penalizes the sum of absolute coefficient values, often setting some coefficients to zero. With correlated predictors, it may select one variable while leaving similar ones out.

Linked from 8 pages