PuzzleLib in a nutshell¶
PuzzleLib - high-level neural network library written in Python. The library is modular and dynamic, its design was inspired by Torch and Chainer. It supports CPU (Intel/AMD) and GPU (NVIDIA/AMD) calculations.
PuzzleLib allows you to:
- build a variety of neural network architectures fast and easy;
- use already implemented neural network architectures, such as
- train and work on both the GPU and CPU.
Installing the library and dependencies¶
In general PuzzleLib supports:
- NVIDIA GPU (CUDA backend);
- AMD GPU (ROCm backend);
- Intel CPU (mkl-dnn backend);
- AMD CPU (mkl-dnn/numpy backend);
- Elbrus CPU (partially)(numpy backend).
Neural networks consist of various modules, that perform certain operations (such as convolution, pooling, batch normalization, etc.). Creating them in PuzzleLib is quite simple:
from PuzzleLib.Modules improt Linear, Activation from PuzzleLib.Modules.Activation import relu lin1 = Linear(insize=16, outsize=64) act = Activation(activation=relu) lin2 = Linear(insize=64, outsize=1)
from PuzzleLib.Containers import Sequential seq = Sequential()
You can add new layers to the container with the
seq.append(lin1) seq.append(act) seq.append(lin2)
There is a more convenient method though - the Graph container. In this case, we would first need to set the dependency between the layers explicitly using the
from PuzzleLib.Containers import Graph lin1 = lin1.node() act = act.node(lin1) lin2 = lin2.node(act) graph = Graph(inputs=lin1, outputs=lin2)
Once we have built the desired model architecture, we should prepare it for training.
from PuzzleLib.Optimizers.MomentumSGD import MomentumSGD optimizer = MomentumSGD() optimizer.setupOn(graph, useGlobalState=True) optimizer.learnRate = 0.1 optimizer.momRate = 0.9
from PuzzleLib.Cost import Abs cost = Abs()
The library provides various handlers - for user's convenience - auxiliary objects for training, validation, and standard calculation of the neural network output.
We create a trainer to start the training process:
from PuzzleLib.Handlers import Trainer, Validator trainer = Trainer(graph, cost, optimizer)
And a validator for the network validation:
from PuzzleLib.Handlers import Trainer validator = Validator(graph, cost)
Please note that the validator does not use an optimizer, unlike the trainer.
And finally, we launch the training on 15 epochs:
import numpy as np data = np.random.randn(10000, 16).astype(np.float32) labels = np.random.randn(10000, 1).astype(np.float32) for i in range(15): trainer.trainFromHost(data[:8000], labels[:8000], macroBatchSize=400, onMacroBatchFinish=lambda train: print("Train error: %s" % train.cost.getMeanError())) print("Accuracy: %s" % (1.0 - validator.validateFromHost(data[8000:], labels[8000:], macroBatchSize=400))) optimizer.learnRate *= 0.9
fromHost prefix means that the data we forward to the
trainFromHost method is stored in the CPU's RAM, and must be first loaded onto the GPU. If the data is already stored there, you can use the similar
macroBatchSize parameter specifies how many objects from the entire selection we will load on the GPU at a time. The input data will be divided into batches, which will be sent to the neural network for training.
We specify which function to execute when the
macroBatch passes through the network in the