SVM

Description

The SVM loss function, which is an extension of the classical Hinge-loss to multiclass classification problems, implements the “one against all” principle.

For the error function, two formulas are possible depending on the chosen metric (L1 or L2):

\begin{equation} L_{L1} = \sum_{y \neq i}max(0, p_y-p_i+1) \end{equation} \begin{equation} L_{L2} = \sum_{y \neq i}max(0, p_y-p_i+1)^2 \end{equation}

where

p_y - "raw" prediction of the classifier regarding the belonging of the object to the real class y;
p_i - "raw" prediction of the classifier regarding the belonging of the object to the i-th class, which is not the target one.

Initializing

def __init__(self, mode="l1"):

Parametrs

Parameter Allowed types Description Default
mode str Selection of the metric by which the error function will be calculated 'l1'

Explanations

mode - takes the values 'l1' or 'l2'.

Examples


Necessary imports:

>>> import numpy as np
>>> from PuzzleLib.Backend import gpuarray
>>> from PuzzleLib.Cost import SVM

Info

gpuarray is required to properly place the tensor in the GPU.

Synthetic target and prediction tensors:

>>> scores = gpuarray.to_gpu(np.random.randn(10, 4).astype(np.float32))
>>> labels = gpuarray.to_gpu((np.random.randint(low=0, high=2, size=(10, )) * 2 - 1).astype(np.int32))

Important

Please remember that the first dimension of target and prediction tensors is the size of the batch

Initializing the error function:

>>> svm = SVM()

Calculating the error and the gradient on the batch:

>>> error, grad = svm(scores, labels)