Parallel¶

Description¶

A container that implements parallel connection (branching) of modules.

Initialization¶

def __init__(self, name=None)

Parameters

Parameter	Allowed types	Description	Default
name	str	container name	None

Explanations

-

Examples¶

Simple parallelization¶

The following network should be implemented:

A branch consisting of two modules:
1.1 Multiplication of the tensor by a constant
1.2 Activation("ReLU")
A transpose module parallel to the first branch
Sum of the results from the two branches

Necessary imports:

>>> import numpy as np
>>> from PuzzleLib.Backend import gpuarray
>>> from PuzzleLib.Containers import Sequential, Parallel
>>> from PuzzleLib.Modules import MulAddConst, Activation, Transpose, Replicate, Add

Info

gpuarray is necessary for the correct placement of the tensor in the GPU

Initialization of the base sequence:

>>> net = Sequential()

The following operation creates the required number of copies of the input data to transfer it to parallel network branches:

>>> net.append(Replicate(times=2, name="replicate"))

A function to create the first branch:

def branch1(name):
  branch = Sequential(name=name)

  branch.append(MulAddConst(a=2, b=0, name="mul_const"))
  branch.append(Activation(activation="relu", name="act"))

  return branch

Creating a parallel container and adding the first branch to it:

>>> parallel_ = Parallel(name="parallel")
>>> parallel_.append(branch1("branch"))

Adding a second branch consisting of one transpose module:

>>> parallel_.append(Transpose(axes=(0, 1, 3, 2), name="transpose"))

Attaching a parallel container to the network and adding a summing module to the output of the parallel section:

>>> net.append(parallel_)
>>> net.append(Add(name="add"))

Synthetic tensor:

>>> data = np.random.randint(-5, 5, size=(1, 1, 3, 3)).astype(np.float32)
>>> print(data)
[[[[-3.  0. -4.]
   [ 0. -5.  0.]
   [ 4. -3. -1.]]]]

Important

For the library to work with tensors correctly, two conditions must be met:

The shape of the input tensor (N, C, H, W), where N is the tensor number in the batch, C is the number of channels (maps), H is the height, W is the width
Tensor data type - float32

Placement of the initial tensor on the GPU and its running through the sequence:

>>> net(gpuarray.to_gpu(data))

Tip

Any element of a sequence can be addressed either by its name or by its index

Result:

>>> # parallel.transpose layer results
>>> print(net["parallel"]["transpose"].data)
[[[[-3.  0.  4.]
   [ 0. -5. -3.]
   [-4.  0. -1.]]]]
>>> # parallel.branch.mul_const layer results
>>> print(net["parallel"]["branch"]["mul_const"].data)
[[[[ -6.   0.  -8.]
   [  0. -10.   0.]
   [  8.  -6.  -2.]]]]
>>> # parallel.branch.act layer results
>>> print(net["parallel"]["branch"]["act"].data)
[[[[0. 0. 0.]
   [0. 0. 0.]
   [8. 0. 0.]]]]
>>> # add layer results
>>> print(net["add"].data)
[[[[-3.  0.  4.]
   [ 0. -5. -3.]
   [ 4.  0. -1.]]]]