Parallel

Description

A container that implements parallel connection (branching) of modules.

Initialization

def __init__(self, name=None)

Parameters

Parameter Allowed types Description Default
name str container name None

Explanations

-

Examples

Simple parallelization

The following network should be implemented:

  1. A branch consisting of two modules:
    1.1 Multiplication of the tensor by a constant
    1.2 Activation("ReLU")

  2. A transpose module parallel to the first branch

  3. Sum of the results from the two branches

Necessary imports: 

import numpy as np
from PuzzleLib.Backend import gpuarray
from PuzzleLib.Containers import Sequential, Parallel
from PuzzleLib.Modules import MulAddConst, Activation, Transpose, Replicate, Add

Info

gpuarray is necessary for the correct placement of the tensor in the GPU

Initialization of the base sequence: 

net = Sequential()

The following operation creates the required number of copies of the input data to transfer it to parallel network branches:

net.append(Replicate(times=2, name="replicate"))

A function to create the first branch: 

def branch1(name):
  branch = Sequential(name=name)

  branch.append(MulAddConst(a=2, b=0, name="mul_const"))
  branch.append(Activation(activation="relu", name="act"))

  return branch

Creating a parallel container and adding the first branch to it: 

parallel_ = Parallel(name="parallel")
parallel_.append(branch1("branch"))

Adding a second branch consisting of one transpose module: 

parallel_.append(Transpose(axes=(0, 1, 3, 2), name="transpose"))

Attaching a parallel container to the network and adding a summing module to the output of the parallel section: 

net.append(parallel_)
net.append(Add(name="add"))

Synthetic tensor: 

np.random.seed(123)
data = np.random.randint(-5, 5, size=(1, 1, 3, 3)).astype(np.float32)
print(data)

[[[[-3. -3.  1.]
   [-4. -2.  4.]
   [ 1. -4. -5.]]]]

Important

For the library to work with tensors correctly, two conditions must be met:

  • The shape of the input tensor (N, C, H, W), where N is the tensor number in the batch, C is the number of channels (maps), H is the height, W is the width
  • Tensor data type - float32

Placement of the initial tensor on the GPU and its running through the sequence: 

net(gpuarray.to_gpu(data))

Tip

Any element of a sequence can be addressed either by its name or by its index

Result: 

# parallel.transpose layer results
print(net["parallel"]["transpose"].data)

[[[[-3. -4.  1.]
   [-3. -2. -4.]
   [ 1.  4. -5.]]]]

# parallel.branch.mul_const layer results
print(net["parallel"]["branch"]["mul_const"].data)

[[[[ -6.  -6.   2.]
   [ -8.  -4.   8.]
   [  2.  -8. -10.]]]]

# parallel.branch.act layer results
print(net["parallel"]["branch"]["act"].data)

[[[[0. 0. 2.]
   [0. 0. 8.]
   [2. 0. 0.]]]]

# add layer results
print(net["add"].data)

[[[[-3. -4.  3.]
   [-3. -2.  4.]
   [ 3.  4. -5.]]]]