Nonlinear least squares

Nonlinear least squares estimates model parameters by minimizing the sum of squared differences between observed data and model predictions when the model is nonlinear in its parameters.

Residual (statistics): A residual is the difference between an observed value and the value predicted by a fitted model. The objective squares and sums these prediction errors.
Least squares: A family of methods that fits parameters by minimizing the sum of squared discrepancies between data and model predictions. Nonlinear least squares is its parameter-nonlinear form.
Curve fitting: The process of choosing a mathematical function to represent observed data. Many curve-fitting tasks estimate parameters in nonlinear functions.
Identifiability: A property describing whether distinct parameter values can be distinguished from the available observations. A small residual does not ensure that fitted parameters are uniquely determined.
Jacobian matrix: A Jacobian matrix contains the first derivatives of a vector-valued function with respect to its inputs. Its columns describe how residuals change with each parameter.
Parameter estimation: The process of inferring unknown model parameters from observed data. Nonlinear least squares is one criterion for estimating parameters.
Exponential decay: A process in which a quantity decreases at a rate proportional to its current value. Estimating its amplitude and rate commonly produces a nonlinear least-squares problem.
Condition number: A quantity that measures how sensitive a problem’s solution is to small changes in its input. Poor conditioning can make parameter estimates highly sensitive to data perturbations.
Gauss–Newton algorithm: An iterative optimization method that approximates a nonlinear least-squares objective using a linearized residual model. It turns the current residuals and Jacobian into a parameter update.
Nonlinear model: A model whose predictions are not linear functions of its variables or parameters, depending on the specified sense. The key distinction is nonlinearity in the parameters being estimated.

Linked from 2 pages