Introduction of Support Vector Machine (SVM)

Generally speaking, SVM (Support Vector Machine) is an algorithm used for data classification and regression. It was developed by Vladimir Vapnik and his colleagues [1] in 1995, based on statistical learning theory. In Geometry perspective, data can be represented as a set of feature vectors in n-dimension space. According to the attributes of training vector sets, SVM splits them into different feature spaces and generates a hyperplane which is a decision function in fact, to predict new unclassified data into the classes they should belong to. Optimal classification requires not only the correct separation but also the maximization of the separation distance (margin), in order to satisfy the structural risk minimization (SRM) notion. SVM settles the problem from the simplest one, which is 2-dimension linear classification. However, for most complex data, it is hard to find the linear hyperplane in low-dimension therefore they are mapped into higher dimension, which will increase the computational complexity dramatically. Therefore, Kernel function is introduced, in order to reduce the computation problem. Hence, proper choice of kernel function helps getting the classification function in higher dimension classifier much easier.

[1] V.N. Vapnik, “The Nature of Statistical LearningTheory,” Springer-Verlag, NewYork, ISBN 0-387-94559-8, 1995.