Orthogonal projection

Orthogonal projection maps a vector to its closest point in a subspace under the Euclidean distance. The difference between the vector and its projection is perpendicular to the subspace.

Orthogonal complement: The subspace of vectors perpendicular to every vector in a given subspace. The projection residual lies in this subspace.
Inner product space: A vector space equipped with an inner product that defines lengths and angles. Orthogonality and distance depend on this structure.
Least squares: A method that chooses parameters to minimize the sum of squared residuals in an overdetermined system. Its fitted vector is the projection of the data onto a model's column space.
Oblique projection: A projection onto a subspace along a direction that is not perpendicular to that subspace. It shares the same idempotent structure but not the minimum-distance property.
Projection matrix: A matrix that maps vectors to their projections onto a specified subspace. Its columns and algebra encode the projection map.
Subspace: A subset of a vector space that is itself a vector space under the same operations. The target of an orthogonal projection must be a subspace.
Linear regression: A statistical method that models a response as a linear combination of predictor variables. Ordinary least-squares regression projects observed responses onto the predictors' span.
Affine projection: The closest-point map onto an affine subspace, which may not pass through the origin. Its target is shifted from a linear subspace, though the residual remains perpendicular.
Gram–Schmidt process: An algorithm that converts a linearly independent list into an orthogonal or orthonormal list spanning the same subspace. It supplies orthonormal directions for computing the projection.
Linear independence: A property of vectors for which only the all-zero linear combination equals zero. Independent spanning vectors give a full-rank basis for projection formulas.

Linked from 19 pages