Hessian matrix

The Hessian matrix is the square matrix of second-order partial derivatives of a scalar-valued function. It describes local curvature and how the function’s gradient changes.

Second partial derivative: A partial derivative taken twice, or taken successively with respect to two variables. Each Hessian entry is a second partial derivative.
Second derivative test: A method for classifying critical points using second derivatives. The multivariable test examines Hessian definiteness at a critical point.
Partial derivative: A derivative measuring how a function changes with respect to one variable while other variables are held fixed. Partial derivatives are the operations applied twice to form Hessian entries.
Laplacian: The sum of a function's pure second partial derivatives, often defined as the divergence of its gradient. Unlike the full Hessian, it reduces curvature information to a scalar.
Mixed partial derivative: A partial derivative taken successively with respect to two different variables. Off-diagonal entries measure interactions between distinct directions.
Newton's method: An iterative method that uses local linear or quadratic approximations to solve equations or optimize functions. Newton optimization steps depend on solving a system involving the Hessian.
Jacobian matrix: The matrix of first-order partial derivatives of a vector-valued function. The Hessian is the Jacobian of the gradient vector.
Gauss–Newton method: An optimization method approximating the Hessian of a least-squares objective using its residual Jacobian. It replaces the exact Hessian with a structured approximation.
Gradient: The vector of first partial derivatives of a scalar-valued function. The Hessian is the Jacobian matrix of the gradient.
Trust-region method: An optimization method that minimizes a local model within a bounded neighborhood of the current point. Its quadratic model commonly uses Hessian curvature.

Linked from 24 pages