Convex optimization

Convex optimization minimizes a convex function over a convex feasible set. Every local minimum is a global minimum, making these problems especially amenable to reliable algorithms and analysis.

Convex set: A set containing every line segment between any two of its points. The feasible region must have this shape for the problem to be convex.
Gradient descent: An iterative method that updates variables opposite the gradient of an objective. It solves smooth unconstrained convex problems through repeated local steps.
Linear programming: Optimization of a linear objective subject to linear constraints. It is the foundational special case of convex optimization.
Nonconvex optimization: Optimization problems whose objective or feasible region is not convex. Local minima need not be global, unlike in convex optimization.
Weak duality: The principle that a dual optimization value bounds the corresponding primal value. It gives certificates and bounds even when strong duality fails.
Convex function: A function whose value along any line segment is no greater than the linear interpolation of its endpoint values. Convex objectives ensure local minima are global.
Projected gradient method: An iterative method that takes gradient steps and projects back onto a feasible set. Projection keeps each iterate feasible while decreasing the objective.
Quadratic programming: Optimization of a quadratic objective subject to linear constraints. Convex quadratic programs arise in control, finance, and estimation.
Integer programming: Optimization with some or all decision variables restricted to integer values. Discrete constraints generally destroy convexity and tractability.
Slater's condition: A strict-feasibility condition that guarantees strong duality for many convex problems. It identifies when Lagrange duality loses no optimal value.

Linked from 24 pages