Principal component analysis

Principal component analysis (PCA) transforms correlated variables into uncorrelated linear combinations called principal components, ordered by the variance they explain. It is commonly used to summarize high-dimensional data and reveal major patterns.

Covariance matrix: A square matrix containing the variances of variables and the covariances between each pair. PCA uses this matrix to represent how the input variables vary together.
Variance: A measure of how far numerical observations spread around their mean. PCA ranks components by the variance of the projected data.
Dimensionality reduction: The process of representing data with fewer variables while retaining selected structure or information. PCA reduces the number of coordinates by keeping only leading components.
Factor analysis: A statistical model that explains observed variable correlations through a smaller set of latent factors and unique errors. Unlike PCA's variance decomposition, factor analysis models shared covariance and variable-specific error.
Explained variance: The portion of a dataset's total variance accounted for by a model or component. It quantifies how much variation each retained PCA component represents.
Eigenvalue: A scalar that describes how a linear transformation scales a corresponding eigenvector. In covariance-based PCA, each eigenvalue measures the variance captured by its component.
Linear combination: An expression formed by multiplying vectors or quantities by scalars and adding the results. Each principal component is a weighted linear combination of the original variables.
Principal component regression: A regression method that predicts an outcome using selected principal component scores as predictors. It uses PCA to reduce predictor dimensionality before fitting a regression model.
Independent component analysis: A method for separating multivariate signals into statistically independent components. ICA seeks independence, whereas PCA guarantees only uncorrelated component scores.
Reconstruction error: A measure of the difference between observed data and an approximation reconstructed from a model. Discarding components trades a simpler representation for error in reconstructing the original observations.

Linked from 28 pages