Dropout2D

Description

Info

Parent class: Dropout

Derived classes: -

This module, like Dropout performs the operation of excluding neurons, but on a larger scale: not individual elements of the features are excluded, but whole maps. That is, roughly speaking, for a tensor of shape $(N, C, H, W)$, where $N$ - batch size, $C$ - number of maps (channels), $H$ - map height $W$ - map width, a certain number of maps will be nullified.

Initializing

def __init__(self, p=0.5, rng=globalRng, slicing=None, inplace=False, name=None):

Parameters

Parameter	Allowed types	Description	Default
p	float	Probability of excluding neurons	0.5
rng	object	Random number generator object (backend-dependent)	globalRng
slicing	slice	Slice, to which the operation is applied	None
inplace	bool	If True, the output tensor will be written in memory in the place of the input tensor	False
name	str	Layer name	None

Explanations

rng - random number generator, which depends on the platform where the calculations are performed, i.e. the generator is backend-dependent: for example, when using Nvidia-GPU and CUDA, the generator is formed using the curand library;

---

slicing - slice is set through the Python built-in slice object. Please have in mind that the slice indexes are set for the flattened data tensor;

---

inplace - flag showing whether additional memory resources should be allocated for the result. If True, then the output tensor will be written in the place of the input tensor in memory, which can negatively affect the network, if the input tensor takes part in calculations on other branches of the graph.

Examples

Necessary imports.

import numpy as np
from PuzzleLib.Backend import gpuarray
from PuzzleLib.Modules import Dropout2D

Info

gpuarray is required to properly place the tensor in the GPU.

batchsize, maps, h, w = 1, 4, 3, 3
data = gpuarray.to_gpu(np.random.randint(1, 9, (batchsize, maps, h, w)).astype(np.float32))
print(data)

[[[[7. 6. 7.]
   [3. 5. 3.]
   [7. 2. 4.]]

  [[3. 4. 2.]
   [7. 2. 1.]
   [2. 7. 8.]]

  [[2. 1. 7.]
   [1. 8. 2.]
   [4. 7. 6.]]

  [[5. 1. 1.]
   [5. 2. 8.]
   [4. 3. 5.]]]]

Let us initialize the module with default parameters:

dropout2d = Dropout2D()
print(dropout2d(data))

[[[[14. 12. 14.]
   [ 6. 10.  6.]
   [14.  4.  8.]]

  [[ 0.  0.  0.]
   [ 0.  0.  0.]
   [ 0.  0.  0.]]

  [[ 4.  2. 14.]
   [ 2. 16.  4.]
   [ 8. 14. 12.]]

  [[ 0.  0.  0.]
   [ 0.  0.  0.]
   [ 0.  0.  0.]]]]