Dropout2D

Description

Info

Parent class: Dropout

Derived classes: -

This module, like Dropout performs the operation of excluding neurons, but on a larger scale: not individual elements of the features are excluded, but whole maps. That is, roughly speaking, for a tensor of shape (N, C, H, W), where N - batch size, C - number of maps (channels), H - map height W - map width, a certain number of maps will be nullified.

Initializing

def __init__(self, p=0.5, rng=globalRng, slicing=None, inplace=False, name=None):

Parameters

Parameter Allowed types Description Default
p float Probability of excluding neurons 0.5
rng object Random number generator object (backend-dependent) globalRng
slicing slice Slice, to which the operation is applied None
inplace bool If True, the output tensor will be written in memory in the place of the input tensor False
name str Layer name None

Explanations

rng - random number generator, which depends on the platform where the calculations are performed, i.e. the generator is backend-dependent: for example, when using Nvidia-GPU and CUDA, the generator is formed using the curand library;


slicing - slice is set through the Python built-in slice object. Please have in mind that the slice indexes are set for the flattened data tensor;


inplace - flag showing whether additional memory resources should be allocated for the result. If True, then the output tensor will be written in the place of the input tensor in memory, which can negatively affect the network, if the input tensor takes part in calculations on other branches of the graph.

Examples

Necessary imports.

>>> import numpy as np
>>> from PuzzleLib.Backend import gpuarray
>>> from PuzzleLib.Modules import Dropout2D

Info

gpuarray is required to properly place the tensor in the GPU.

>>> batchsize, maps, h, w = 1, 4, 3, 3
>>> data = gpuarray.to_gpu(np.random.randint(1, 9, (batchsize, maps, h, w)).astype(np.float32))
>>> data
[[[[6. 6. 2.]
   [8. 6. 7.]
   [2. 5. 8.]]

  [[1. 5. 5.]
   [2. 1. 3.]
   [1. 4. 4.]]

  [[8. 5. 2.]
   [8. 5. 7.]
   [5. 4. 4.]]

  [[7. 6. 5.]
   [3. 4. 4.]
   [1. 4. 2.]]]]

Let us initialize the module with default parameters:

>>> dropout2d = Dropout2D()
[[[[12. 12.  4.]
   [16. 12. 14.]
   [ 4. 10. 16.]]

  [[ 2. 10. 10.]
   [ 4.  2.  6.]
   [ 2.  8.  8.]]

  [[ 0.  0.  0.]
   [ 0.  0.  0.]
   [ 0.  0.  0.]]

  [[14. 12. 10.]
   [ 6.  8.  8.]
   [ 2.  8.  4.]]]]