PuzzleLib in a nutshell¶

Introduction¶

PuzzleLib - high-level neural network library written in Python. The library is modular and dynamic, its design was inspired by Torch and Chainer. It supports CPU (Intel/AMD) and GPU (NVIDIA/AMD) calculations.

PuzzleLib allows you to:

build a variety of neural network architectures fast and easy;
use already implemented neural network architectures, such as VGG, ResNet, Inception, UNet and others;
train and work on both the GPU and CPU.

Installing the library and dependencies¶

For detailed instructions on how to install the library and dependencies, please see the documentation for installation on Windows or Linux systems.

In general PuzzleLib supports:

NVIDIA GPU (CUDA backend);
AMD GPU (ROCm backend);
Intel CPU (mkl-dnn backend);
AMD CPU (mkl-dnn/numpy backend);
Elbrus CPU (partially)(numpy backend).

Functional¶

Neural networks consist of various modules, that perform certain operations (such as convolution, pooling, batch normalization, etc.). Creating them in PuzzleLib is quite simple:

from PuzzleLib.Modules improt Linear, Activation
from PuzzleLib.Modules.Activation import relu

lin1 = Linear(insize=16, outsize=64)
act = Activation(activation=relu)
lin2 = Linear(insize=64, outsize=1)

However, the network blocks must be interconnected, which requires containers, such as Sequential:

from PuzzleLib.Containers import Sequential

seq = Sequential()

You can add new layers to the container with the append command:

seq.append(lin1)
seq.append(act)
seq.append(lin2)

There is a more convenient method though - the Graph container. In this case, we would first need to set the dependency between the layers explicitly using the node method:

from PuzzleLib.Containers import Graph

lin1 = lin1.node()
act = act.node(lin1)
lin2 = lin2.node(act)

graph = Graph(inputs=lin1, outputs=lin2)

Once we have built the desired model architecture, we should prepare it for training.

To do this, we need to select a fitting training algorithm implemented in the Optimizers block, for example, MomentumSGD. Now setup the optimizer on the net.

from PuzzleLib.Optimizers.MomentumSGD import MomentumSGD

optimizer = MomentumSGD()
optimizer.setupOn(graph, useGlobalState=True)
optimizer.learnRate = 0.1
optimizer.momRate = 0.9

You should also define a loss function, that will evaluate how wrong the network was when classifying an input object. Suppose, it's Abs:

from PuzzleLib.Cost import Abs

cost = Abs()

The library provides various handlers - for user's convenience - auxiliary objects for training, validation, and standard calculation of the neural network output.

We create a trainer to start the training process:

from PuzzleLib.Handlers import Trainer, Validator

trainer = Trainer(graph, cost, optimizer)

And a validator for the network validation:

from PuzzleLib.Handlers import Trainer

validator = Validator(graph, cost)

Please note that the validator does not use an optimizer, unlike the trainer.

And finally, we launch the training on 15 epochs:

import numpy as np

data = np.random.randn(10000, 16).astype(np.float32)
labels = np.random.randn(10000, 1).astype(np.float32)

for i in range(15):
    trainer.trainFromHost(data[:8000], labels[:8000], macroBatchSize=400,
                            onMacroBatchFinish=lambda train: print("Train error: %s" % train.cost.getMeanError()))
    print("Accuracy: %s" % (1.0 - validator.validateFromHost(data[8000:], labels[8000:], macroBatchSize=400)))

    optimizer.learnRate *= 0.9

The fromHost prefix means that the data we forward to the trainFromHost method is stored in the CPU's RAM, and must be first loaded onto the GPU. If the data is already stored there, you can use the similar train method.

The macroBatchSize parameter specifies how many objects from the entire selection we will load on the GPU at a time. The input data will be divided into batches, which will be sent to the neural network for training.

We specify which function to execute when the macroBatch passes through the network in the onMacroBatchFinish parameter.