Hook is the base class for a family of called objects that must perform certain operations on gradients before updating the weights. Examples of possible applications for hooks: normalizing the gradient, chopping off at a fixed threshold, adding noise to the gradient, etc.

Creating a custom Hook

To implement a custom hook, you need to declare a method __call__:

def call(self, var, state, stream=None):


Parameter Allowed types Description Default
var Variable Object of the Variable library class, which is a wrapper around the GPU tensors -
state str String indicator of var belonging to a particular data type -
stream - - None