Hook is the base class for a family of called objects that must perform certain operations on gradients before updating the weights. Examples of possible applications for hooks: normalizing the gradient, chopping off at a fixed threshold, adding noise to the gradient, etc.
Creating a custom Hook¶
To implement a custom hook, you need to declare a method
def call(self, var, state, stream=None):
|var||Variable||Object of the Variable library class, which is a wrapper around the GPU tensors||-|
|state||str||String indicator of var belonging to a particular data type||-|