Parallel¶
Description¶
A container that implements parallel connection (branching) of modules.
Initialization¶
def __init__(self, name=None)
Parameters
Parameter | Allowed types | Description | Default |
---|---|---|---|
name | str | container name | None |
Explanations
-
Examples¶
Simple parallelization¶
The following network should be implemented:
-
A branch consisting of two modules:
1.1 Multiplication of the tensor by a constant
1.2 Activation("ReLU") -
A transpose module parallel to the first branch
- Sum of the results from the two branches
Necessary imports:
>>> import numpy as np
>>> from PuzzleLib.Backend import gpuarray
>>> from PuzzleLib.Containers import Sequential, Parallel
>>> from PuzzleLib.Modules import MulAddConst, Activation, Transpose, Replicate, Add
Info
gpuarray
is necessary for the correct placement of the tensor in the GPU
Initialization of the base sequence:
>>> net = Sequential()
The following operation creates the required number of copies of the input data to transfer it to parallel network branches:
>>> net.append(Replicate(times=2, name="replicate"))
A function to create the first branch:
def branch1(name):
branch = Sequential(name=name)
branch.append(MulAddConst(a=2, b=0, name="mul_const"))
branch.append(Activation(activation="relu", name="act"))
return branch
Creating a parallel container and adding the first branch to it:
>>> parallel_ = Parallel(name="parallel")
>>> parallel_.append(branch1("branch"))
Adding a second branch consisting of one transpose module:
>>> parallel_.append(Transpose(axes=(0, 1, 3, 2), name="transpose"))
Attaching a parallel container to the network and adding a summing module to the output of the parallel section:
>>> net.append(parallel_)
>>> net.append(Add(name="add"))
Synthetic tensor:
>>> data = np.random.randint(-5, 5, size=(1, 1, 3, 3)).astype(np.float32)
>>> print(data)
[[[[-3. 0. -4.]
[ 0. -5. 0.]
[ 4. -3. -1.]]]]
Important
For the library to work with tensors correctly, two conditions must be met:
- The shape of the input tensor (N, C, H, W), where N is the tensor number in the batch, C is the number of channels (maps), H is the height, W is the width
- Tensor data type - float32
Placement of the initial tensor on the GPU and its running through the sequence:
>>> net(gpuarray.to_gpu(data))
Tip
Any element of a sequence can be addressed either by its name or by its index
Result:
>>> # parallel.transpose layer results
>>> print(net["parallel"]["transpose"].data)
[[[[-3. 0. 4.]
[ 0. -5. -3.]
[-4. 0. -1.]]]]
>>> # parallel.branch.mul_const layer results
>>> print(net["parallel"]["branch"]["mul_const"].data)
[[[[ -6. 0. -8.]
[ 0. -10. 0.]
[ 8. -6. -2.]]]]
>>> # parallel.branch.act layer results
>>> print(net["parallel"]["branch"]["act"].data)
[[[[0. 0. 0.]
[0. 0. 0.]
[8. 0. 0.]]]]
>>> # add layer results
>>> print(net["add"].data)
[[[[-3. 0. 4.]
[ 0. -5. -3.]
[ 4. 0. -1.]]]]