Penalty

Description

Info

Parent class: Module

Derived classes: -

This module implements the L1 (Lasso) or L2 (Ridge) regularization function.

It is used to reduce the likelihood of overfitting the model.

Additional sources

• Towardsdatascience

Initializing

def __init__(self, mode="l1", weight=1e-2, name=None):

Parameters

Parameter	Allowed types	Description	Default
mode	str	Regularization type, can be "l1"or "l2"	"l1"
weight	float	Regularization parameter value	1e-2
name	str	Layer name	None

Explanations

-

Examples

import numpy as np
from PuzzleLib.Backend import gpuarray
from PuzzleLib.Modules import Penalty

data = gpuarray.to_gpu(np.random.randn(10, 50).astype(np.float32))

penalty = Penalty()
penalty(data)

grad = gpuarray.to_gpu(np.random.randn(10, 50).astype(np.float32))
penalty.backward(grad)