Large Margin Classification

Optimisation Objective

  • Alternative View Of Logistic Regression

    • If y = 1, h(x) should be similar to 1, theta^T x » 0

    • If y = 0, h(x) should be similar to 0, theta^T x » 1

    Alternate View LOR.png

    • Graph plotting of function z

    Alternate View LOR 1.png

  • Support Vector Machine

    • Modified hypothesis of Logistic Regression

    SVM.png SVM Hypothesis.png

Large Margin Intuition

  • Concept

    SVM Concept.png

  • SVM Decision Boundary

    SVM Decision Boundary.png

    • SVM creates a decision boundary in a way that there is some space between the samples and decision boundary

    • That space is called as margin

    SVM Decision Boundary 1.png

    • If C is too large the decision boundary will identical to magenta line

    • If C is not to large then the decision boundary will be identical to black line

    SVM Decision Boundary 2.png

Mathematics Behind Large Margin Classification

  • Vector Inner Product

    •   u   → euclidean length of vector u
    • p → length of projection of v onto u

    Vector Inner Product.png

  • SVM Decision Boundary

    SVM DB.png SVM DB 1.png SVM DB 2.png

Kernels

Kernels I

  • Non Linear Decision Boundary

    Non Linear DB.png

  • Kernels

    Kernel.png

    • Similarity

      • Gaussian Kernel

      Kernels and Similarity.png

    • Exmaple

      Example.png

    • Concept

      Concept.png

Kernels II

  • Choosing Landmarks

    Choosing Landmarks.png

  • SVM with Kernels

    SVM with Kernels.png SVM with Kernels 1.png

  • SVM Parameters

    SVM Parameters.png

SVMs in Practice

Using An SVM

  • Overview

    Overview.png

  • Octave Implementation

    Octave Implementation.png

  • Other Kernels

    Other Kernels.png

  • Multi Class Classification

    Multi-class Classification.png

  • Logistic Regression VS Support Vector Machine

    LOR vs SVM.png

Lecture Presentation