Back to Curriculum
Intermediate•Machine Learning
SVM
Support Vector Machines for optimal boundary separation.
Interactive Playground
Initializing Interactive Playground...
Research-Level Deep Dive & Equations
The Support Vector Machine (SVM) is a classification framework rooted in convex geometry and statistical learning theory (specifically Vapnik-Chervonenkis theory). Let us consider a binary classification task with training dataset , where represents the feature vector, and denotes the binary class label. We assume initially that the classes are linearly separable in the input space.
We seek a separating hyperplane parameterized by a weight vector and a bias scalar :
The decision rule is given by the sign of the affine transformation: .
**Geometric Margin Derivation**:
The algebraic distance of any point to the hyperplane is the orthogonal projection of the vector (where is any point on the hyperplane) onto the unit normal vector :
Since , we can define the **geometric margin** of the observation as:
The geometric margin of the entire dataset, , is the minimum distance from any training point to the separating hyperplane:
We want to find parameters that maximize this geometric margin . Since the geometric margin is invariant to the scaling of both and by any positive scalar factor, we can introduce a scale normalization constraint. Specifically, we set the functional margin of the closest point to unity:
Under this normalization, the geometric margin is simply:
Maximizing the geometric margin is mathematically identical to minimizing , which we write as the convex quadratic programming (QP) **Primal Problem**:
Key Equations
Test Your Knowledge
Check whether you have mastered this concept with a quick quiz.
Was this lesson helpful?
Your feedback helps us continuously improve the curriculum and interactive visualizations.