SVM

Description

The SVM loss function, which is an extension of the classical Hinge-loss to multiclass classification problems, implements the “one against all” principle.

For the error function, two formulas are possible depending on the chosen metric (L1 or L2):

\begin{equation} L_{L1} = \sum_{y \neq i}max(0, p_y-p_i+1) \end{equation} \begin{equation} L_{L2} = \sum_{y \neq i}max(0, p_y-p_i+1)^2 \end{equation}

where

$p_y$ - "raw" prediction of the classifier regarding the belonging of the object to the real class y;
$p_i$ - "raw" prediction of the classifier regarding the belonging of the object to the i-th class, which is not the target one.

Initializing

def __init__(self, mode="l1"):

Parametrs

Parameter	Allowed types	Description	Default
mode	str	Selection of the metric by which the error function will be calculated	'l1'

Explanations

mode - takes the values 'l1' or 'l2'.

Examples

---

Necessary imports:

import numpy as np
from PuzzleLib.Backend import gpuarray
from PuzzleLib.Cost import SVM

Info

gpuarray is required to properly place the tensor in the GPU.

Synthetic target and prediction tensors:

scores = gpuarray.to_gpu(np.random.randn(10, 4).astype(np.float32))
labels = gpuarray.to_gpu((np.random.randint(low=0, high=2, size=(10, )) * 2 - 1).astype(np.int32))

Important

Please remember that the first dimension of target and prediction tensors is the size of the batch

Initializing the error function:

svm = SVM()

Calculating the error and the gradient on the batch:

error, grad = svm(scores, labels)