Support Vector Machines


SVM Parameters   C “However, it is critical here, as in any regularization scheme, that a proper value is chosen for C, the penalty factor. If it is too large, we have a high penalty for nonseparable points and we may store many support vectors and overfit. If it is too small, we may have …

Parameters Read More »

PAC Learning

PAC Learning   The paper that proposed the PAC learning framework: L. Valiant. A Theory of the Learnable. Communications of the ACM, 27(11):1134–1142, 1984. Communications of the ACM Volume 27 , Issue 11 (November 1984) Pages: 1134 – 1142 Year of Publication: 1984 Author: L. G. Valiant Links Wikipedia: Probably approximately correct learning Bibliography AIZENSTEIN, …

PAC Learning Read More »


nu   SCH?O?LKOPF, Bernhard, et al., New Support Vector Algorithms “We describe a new class of Support Vector algorithms for regression and classification. In these algorithms, a parameter v lets one effectively control the number of Support Vectors.”


Norm   Norm (mathematics) – Wikipedia Norm — From MathWorld L1-Norm — From MathWorld L2-Norm — From MathWorld “In linear algebra, functional analysis and related areas of mathematics, a norm is a function which assigns a positive length or size to all vectors in a vector space, other than the zero vector.” Wikipedia (2006) “The norm of a mathematical object is …

Norm Read More »

Mercer’s Condition

Mercer’s Condition   Mercer’s condition – Wikipedia “Mercer’s condition” – Google Scholar “Mercer’s condition” – Google Search Finally, what happens if one uses a kernel which does not satisfy Mercer�s condition? In general, there may exist data such that the Hessian is indefinite, and for which the quadratic programming problem will have no solution (the …

Mercer’s Condition Read More »


Learnability   Home ALON, Noga, Scale-Sensitive Dimensions, Uniform Convergence, and Learnability class-12.pdf lfd1.pdf lfd2.pdf MENDELSON, Shahar, Learnability in Hilbert spaces with Reproducing Kernels Support Vector Learning


Lagrangian   Lagrangian – Wikipedia Lagrangian – Google Scholar Lagrangian – Google Search “We will now switch to a Lagrangian formulation of the problem. There are two reasons for doing this. The first is that the constraints (12) will be replaced by constraints on the Lagrange multipliers themselves, which will be much easier to handle. …

Lagrangian Read More »

Karush-Kuhn-Tucker conditions

The Karush-Kuhn-Tucker (KKT) conditions   Karush-Kuhn-Tucker – Wikipedia Karush-Kuhn-Tucker – Google Scholar Karush-Kuhn-Tucker – Google Search “The Karush-Kuhn-Tucker (KKT) conditions play a central role in both the theory and practice of constrained optimization. For the primal problem above, the KKT conditions may be stated (Fletcher, 1987): The KKT conditions are satisfied at the solution of …

Karush-Kuhn-Tucker conditions Read More »