Constrained optimization

Constrained optimization finds minima or maxima of an objective function among points that satisfy specified restrictions, such as equations or inequalities.

Lagrange multiplier: A scalar introduced to combine an objective with an equality constraint in first-order optimality conditions. At a smooth constrained optimum, the objective gradient can align with constraint gradients through these multipliers.
Objective function: A function whose value an optimization problem seeks to minimize or maximize. Its values determine which feasible point counts as best.
Linear programming: Optimization of a linear objective subject to linear equality and inequality constraints. It is the tractable, widely used case where both the objective and restrictions are linear.
Unconstrained optimization: Optimization of an objective function without explicit restrictions on its decision variables. It lacks the feasibility boundaries that can create constrained optima.
Karush–Kuhn–Tucker conditions: First-order conditions for optimality in problems with equality and inequality constraints. They extend multiplier conditions to inequalities, including complementary slackness for inactive constraints.
Equality constraint: A restriction requiring a function of the decision variables to equal a specified value. Equality restrictions often define a lower-dimensional surface on which the search takes place.
Quadratic programming: Optimization of a quadratic objective subject to linear constraints. It captures smooth curved objectives while retaining a structured feasible region.
Convex optimization: Optimization of a convex objective over a convex feasible set, enabling strong global guarantees. Convexity can turn local optimality conditions into global certificates.
Feasible region: The set of points satisfying all constraints of an optimization problem. The optimum must lie in this set, often on its boundary.
Inequality constraint: A restriction requiring a function of the decision variables to lie above or below a specified value. Inequalities create boundaries that may become active at an optimum.

Linked from 17 pages