Node

Info

This module is intended primarily for developers who want to better understand the structure of the library, as well as those who are going to implement their own modules.

Description

A wrapper function around the standard module of the library, which turns it into a node of the network graph. Thus, it becomes possible to write the network using the graph method.

Initializing

def __init__(self, mod, parents=None, name=None):

Parameters

Parameter Allowed types Description Default
mod Module An object of the Module descendant class that defines the functionality of the node -
parents Union[None, Node, tuple, list] Parent nodes None
name str Container name None

Explanations

parents - parent nodes are those nodes of the graph whose output is the input to the current node. Possible options for passing an argument:

  • None - parent nodes are absent;
  • Node - the transmitted node is the only ancestor;
  • tuple - a tuple (Node, idxs), where idxs - single index or a list of indexes of specific outputs of the Node node, that is, if after processing the node returns several data tensors with indexes specified, we can determine which ones will go into further processing;
  • list - a list that is a combination of objects of the two previous types

Methods

addBackwards

def addBackwards(self, nodes):
Functionality

The method is intended for:

  • saving information about the ancestors in the current node;
  • saving information about the descendants in the ancestors. For each object of the Node class, there are fwds and bwds attributes that contain information about subsequent and previous nodes, respectively. The addBackwards method updates the bwds attribute for the current node and the fwds attribute for its ancestors.

Parameters

Parameter Allowed types Description Default
nodes Union[None, Node, tuple, List[tuple]] Parent nodes None

Explanations

nodes - see Initialization

addForward

def addForward(self, nodes):
Functionality

Updates the value of the fwds attribute.

Parameters

Parameter Allowed types Description Default
nodes Union[None, Node, tuple, List[tuple]] Parent nodes None

Explanations nodes - see Initialization

forward

def forward(self, data):
Functionality

It travels along the graph branch for which the current node is the parent, performing operations assigned to the nodes of the graph on the transferred tensor.

Parameters

Parameter Allowed types Description Default
data tensor Data tensor None

Explanations

data - the data tensor has at least two dimensions (for example, fully connected layers), where in the first place is the number of the element in the batch

name

@property
def name(self):

Functionality

Returns the name of the node.

updateData

def updateData(self, data):
Functionality

Applies the operation on the transferred tensor assigned to the node.

Parameters

Parameter Allowed types Description Default
data tensor Data tensor None

Explanations

data - see forward

dataShapeFrom

def dataShapeFrom(self, inshapes, shapes, onmodule):

Functionality

Calculates the shape of the data after its processing by the graph up to the current node. The method works only as an auxiliary one for graph shape methods.

Parameters

Parameter Allowed types Description Default
inshapes dict Dictionary of the input graph nodes shapes -
shapes dict Dictionary of all the graph nodes shapes -
onmodule Callable Additional function -

Explanations

shapes - essentially a container in which information about all calculated shapes is stored; at the very beginning of calculations it is empty, and at the end it contains information about the shapes of all the passed nodes


onmmodule - an additional function that will be used in the calculation of shapes; as a rule, this is a function for optimizing a node for its output shape

backward

def backward(self, grad=None, updParamGrads=True, updGrad=True, scale=1.0, momentum=0.0):

Functionality

It travels along the graph branch, for which the current node is the last descendant, in the opposite direction, implementing the error back propagation algorithm, namely calculating the gradients on the parameters of the nodes and on their input data.

Parameters

Parameter Allowed types Description Default
grad tensor Gradient tensor of the objective function None
updParamGrads bool Flag that regulates the calculation of the gradient on the parameters of the node True
updGrad bool Flag that regulates the calculation of the gradient on the input data of the node True
scale float Scale: determines the scale of the gradient 1.0
momentum float Momentum: determines how much gradient should be kept in this iteration 0.0

Explanations

-

updateGrad

def updateGrad(self, grad, updParamGrads, updGrad, scale, momentum):

Functionality

Calculates the gradients of the parameters and of the input data of the node.

Parameters

Parameter Allowed types Description Default
grad tensor Gradient tensor -
updParamGrads bool Flag that regulates the calculation of the gradient on the parameters of the node True
updGrad bool Flag that regulates the calculation of the gradient on the input data of the node True
scale float Scale: determines the scale of the gradient 1.0
momentum float Momentum: determines how much gradient should be kept in this iteration 0.0

Explanations

-

buildOutGrad

def buildOutGrad(self, grad):

Functionality

Prepares a preliminary list of gradient tensors from the descendant nodes. If there are no descendants, then returns the transferred gradient tensor.

Parameters

Parameter Allowed types Description Default
grad tensor Gradient tensor -

Explanations

-

routeInGrad

def routeInGrad(self, grad):

Functionality

Prepares a preliminary dictionary of gradient tensors/gradient shapes for ancestor nodes. If there are no ancestors, then it returns the transferred gradient tensor/gradient shape.

Parameters

Parameter Allowed types Description Default
grad tensor Gradient tensor -

Explanations

-

gradShapeFrom

def gradShapeFrom(self, outshapes, shapes):

Functionality

Calculates the gradient shape on the current node.

Parameters

Parameter Allowed types Description Default
outshapes tuple Gradient shape from the target function -
shapes dict Dictionary of gradient shapes from all nodes -

Explanations

shapes - a kind of container, which is replenished with shapes when passing through the graph

buildOutGradShape

def buildOutGradShape(self, outshapes, shapes):

Functionality

Prepares a preliminary list of gradient shapes from the descendant nodes. If there are no descendants, then returns the output shape of the node.

Parameters

Parameter Allowed types Description Default
outshapes tuple Gradient shape from the target function -
shapes dict Dictionary of gradient shapes from all nodes -

Explanations

-

reset

def reset(self):

Functionality

Resets the internal state of the node: input data, calculated characteristics, gradients, and also unsets the traverse flags fwdVisited and bwdVisited.

Parameters

-

Explanations

-

clearTraverse

def clearTraverse(self):

Functionality

Unsets the traverse flags fwdVisited and bwdVisited.

Parameters

-

Explanations

-

__str__

def __str__(self):

Functionality

Prepares a readable node name to be displayed.

Parameters

-

Explanations

-

traverseForward

@staticmethod
def traverseForward(self, node, func, *args):

Functionality

Performs a recursive pass through all the derived nodes.

Parameters

Parameter Allowed types Description Default
node Node Node descendant class object -
func Callable Function that performs the required operation on pass -
args list List of positional arguments for the func function -

Explanations

-

traverseBackward

@staticmethod
def traverseBackward(self):

Functionality

Performs a recursive pass through all the parent nodes.

Parameters

Parameter Allowed types Description Default
node Node Node descendant class object -
func Callable Function that performs the required operation on pass -
args list List of positional arguments for the func function -

Explanations

func - for example, Node.updateGrad.