Node¶
Info
This module is intended primarily for developers who want to better understand the structure of the library, as well as those who are going to implement their own modules.
Description¶
A wrapper function around the standard module of the library, which turns it into a node of the network graph. Thus, it becomes possible to write the network using the graph method.
Initializing¶
def __init__(self, mod, parents=None, name=None):
Parameters
| Parameter | Allowed types | Description | Default |
|---|---|---|---|
| mod | Module | An object of the Module descendant class that defines the functionality of the node | - |
| parents | Union[None, Node, tuple, list] | Parent nodes | None |
| name | str | Container name | None |
Explanations
parents - parent nodes are those nodes of the graph whose output is the input to the current node. Possible options for passing an argument:
- None - parent nodes are absent;
- Node - the transmitted node is the only ancestor;
- tuple - a tuple (Node, idxs), where idxs - single index or a list of indexes of specific outputs of the Node node, that is, if after processing the node returns several data tensors with indexes specified, we can determine which ones will go into further processing;
- list - a list that is a combination of objects of the two previous types
Methods¶
addBackwards¶
def addBackwards(self, nodes):
The method is intended for:
- saving information about the ancestors in the current node;
- saving information about the descendants in the ancestors.
For each object of the Node class, there are
fwdsandbwdsattributes that contain information about subsequent and previous nodes, respectively. TheaddBackwardsmethod updates thebwdsattribute for the current node and thefwdsattribute for its ancestors.
Parameters
| Parameter | Allowed types | Description | Default |
|---|---|---|---|
| nodes | Union[None, Node, tuple, List[tuple]] | Parent nodes | None |
Explanations
nodes - see Initialization
addForward¶
def addForward(self, nodes):
Updates the value of the fwds attribute.
Parameters
| Parameter | Allowed types | Description | Default |
|---|---|---|---|
| nodes | Union[None, Node, tuple, List[tuple]] | Parent nodes | None |
Explanations
nodes - see Initialization
forward¶
def forward(self, data):
It travels along the graph branch for which the current node is the parent, performing operations assigned to the nodes of the graph on the transferred tensor.
Parameters
| Parameter | Allowed types | Description | Default |
|---|---|---|---|
| data | tensor | Data tensor | None |
Explanations
data - the data tensor has at least two dimensions (for example, fully connected layers), where in the first place is the number of the element in the batch
name¶
@property
def name(self):
Functionality
Returns the name of the node.
updateData¶
def updateData(self, data):
Applies the operation on the transferred tensor assigned to the node.
Parameters
| Parameter | Allowed types | Description | Default |
|---|---|---|---|
| data | tensor | Data tensor | None |
Explanations
data - see forward
dataShapeFrom¶
def dataShapeFrom(self, inshapes, shapes, onmodule):
Functionality
Calculates the shape of the data after its processing by the graph up to the current node. The method works only as an auxiliary one for graph shape methods.
Parameters
| Parameter | Allowed types | Description | Default |
|---|---|---|---|
| inshapes | dict | Dictionary of the input graph nodes shapes | - |
| shapes | dict | Dictionary of all the graph nodes shapes | - |
| onmodule | Callable | Additional function | - |
Explanations
shapes - essentially a container in which information about all calculated shapes is stored; at the very beginning of calculations it is empty, and at the end it contains information about the shapes of all the passed nodes
onmmodule - an additional function that will be used in the calculation of shapes; as a rule, this is a function for optimizing a node for its output shape
backward¶
def backward(self, grad=None, updParamGrads=True, updGrad=True, scale=1.0, momentum=0.0):
Functionality
It travels along the graph branch, for which the current node is the last descendant, in the opposite direction, implementing the error back propagation algorithm, namely calculating the gradients on the parameters of the nodes and on their input data.
Parameters
| Parameter | Allowed types | Description | Default |
|---|---|---|---|
| grad | tensor | Gradient tensor of the objective function | None |
| updParamGrads | bool | Flag that regulates the calculation of the gradient on the parameters of the node | True |
| updGrad | bool | Flag that regulates the calculation of the gradient on the input data of the node | True |
| scale | float | Scale: determines the scale of the gradient | 1.0 |
| momentum | float | Momentum: determines how much gradient should be kept in this iteration | 0.0 |
Explanations
-
updateGrad¶
def updateGrad(self, grad, updParamGrads, updGrad, scale, momentum):
Functionality
Calculates the gradients of the parameters and of the input data of the node.
Parameters
| Parameter | Allowed types | Description | Default |
|---|---|---|---|
| grad | tensor | Gradient tensor | - |
| updParamGrads | bool | Flag that regulates the calculation of the gradient on the parameters of the node | True |
| updGrad | bool | Flag that regulates the calculation of the gradient on the input data of the node | True |
| scale | float | Scale: determines the scale of the gradient | 1.0 |
| momentum | float | Momentum: determines how much gradient should be kept in this iteration | 0.0 |
Explanations
-
buildOutGrad¶
def buildOutGrad(self, grad):
Functionality
Prepares a preliminary list of gradient tensors from the descendant nodes. If there are no descendants, then returns the transferred gradient tensor.
Parameters
| Parameter | Allowed types | Description | Default |
|---|---|---|---|
| grad | tensor | Gradient tensor | - |
Explanations
-
routeInGrad¶
def routeInGrad(self, grad):
Functionality
Prepares a preliminary dictionary of gradient tensors/gradient shapes for ancestor nodes. If there are no ancestors, then it returns the transferred gradient tensor/gradient shape.
Parameters
| Parameter | Allowed types | Description | Default |
|---|---|---|---|
| grad | tensor | Gradient tensor | - |
Explanations
-
gradShapeFrom¶
def gradShapeFrom(self, outshapes, shapes):
Functionality
Calculates the gradient shape on the current node.
Parameters
| Parameter | Allowed types | Description | Default |
|---|---|---|---|
| outshapes | tuple | Gradient shape from the target function | - |
| shapes | dict | Dictionary of gradient shapes from all nodes | - |
Explanations
shapes - a kind of container, which is replenished with shapes when passing through the graph
buildOutGradShape¶
def buildOutGradShape(self, outshapes, shapes):
Functionality
Prepares a preliminary list of gradient shapes from the descendant nodes. If there are no descendants, then returns the output shape of the node.
Parameters
| Parameter | Allowed types | Description | Default |
|---|---|---|---|
| outshapes | tuple | Gradient shape from the target function | - |
| shapes | dict | Dictionary of gradient shapes from all nodes | - |
Explanations
-
reset¶
def reset(self):
Functionality
Resets the internal state of the node: input data, calculated characteristics, gradients, and also unsets the traverse flags fwdVisited and bwdVisited.
Parameters
-
Explanations
-
clearTraverse¶
def clearTraverse(self):
Functionality
Unsets the traverse flags fwdVisited and bwdVisited.
Parameters
-
Explanations
-
__str__¶
def __str__(self):
Functionality
Prepares a readable node name to be displayed.
Parameters
-
Explanations
-
traverseForward¶
@staticmethod
def traverseForward(self, node, func, *args):
Functionality
Performs a recursive pass through all the derived nodes.
Parameters
| Parameter | Allowed types | Description | Default |
|---|---|---|---|
| node | Node | Node descendant class object | - |
| func | Callable | Function that performs the required operation on pass | - |
| args | list | List of positional arguments for the func function |
- |
Explanations
-
traverseBackward¶
@staticmethod
def traverseBackward(self):
Functionality
Performs a recursive pass through all the parent nodes.
Parameters
| Parameter | Allowed types | Description | Default |
|---|---|---|---|
| node | Node | Node descendant class object | - |
| func | Callable | Function that performs the required operation on pass | - |
| args | list | List of positional arguments for the func function | - |
Explanations
func - for example, Node.updateGrad.