Back to Curriculum
Intermediate•Statistics
Convex Optimization & KKT
Lagrange multipliers, KKT optimality conditions, Slater condition, and duality.
Interactive Playground
Initializing Interactive Playground...
Research-Level Deep Dive & Equations
In mathematical optimization, a set is defined as **convex** if the line segment connecting any two points in lies entirely within .
•Mathematical Definition of a Convex Set:
•Mathematical Definition of a Convex Function: A function is convex if its domain is a convex set and for all and :
•Epigraph & First-Order Characterization: A continuously differentiable function is convex if and only if its epigraph is a convex set, which yields the global first-order lower-bound inequality:
This implies that for a convex function, any local minimum is guaranteed to be a **global minimum**!
•Second-Order Conditions: A twice-differentiable function is convex if and only if its Hessian matrix is positive semi-definite for all :
Key Equations
PyTorch Convexity & Hessian Eigenvalue Inspectorpython
import torch
def inspect_function_convexity(func, x_val: torch.Tensor):
"""Computes the Hessian matrix of a scalar function f(x) and checks positive semi-definiteness."""
x_val = x_val.clone().detach().requires_grad_(True)
y = func(x_val)
# Compute Hessian matrix via autograd
hessian = torch.autograd.functional.hessian(func, x_val)
eigenvalues = torch.linalg.eigvalsh(hessian)
is_convex = torch.all(eigenvalues >= -1e-6)
print(f"Hessian Eigenvalues at x={x_val.tolist()}: {eigenvalues.tolist()}")
print(f"Is function locally convex? {is_convex}")
return is_convex
# Example: Quadratic objective f(x) = x1^2 + 3*x2^2 + 2*x1*x2
f_quadratic = lambda x: x[0]**2 + 3*x[1]**2 + 2*x[0]*x[1]
inspect_function_convexity(f_quadratic, torch.tensor([1.0, 2.0]))Test Your Knowledge
Check whether you have mastered this concept with a quick quiz.
Was this lesson helpful?
Your feedback helps us continuously improve the curriculum and interactive visualizations.