MaxPool2D

Description

Info

Parent class: Pool2D

Derived classes: -

This module implements the operation of two-dimensional max pooling. For a detailed theoretical description, please see Pool2D.

For an input tensor of shape (N, C, H_{in}, W_{in}) and the output one of shape (N, C, H_{out}, W_{out}) the operation is performed as follows (we consider the i-th element of the batch and the j-th map of the output tensor):

out(N_i, C_j, h, w) = \max\limits_{m=0..k_h-1}\max\limits_{n=0..k_w-1}(input(N_i, C_j, stride_h \times h + m, stride_w \times w + n))

where

N - size of the batch;
C - number of maps in the tensor;
H - tensor map height;
W - tensor map width;
stride_h, stride_w - pooling stride along the depth, height and width of the maps, respectively;
k_h, k_w - pooling kernel size in depth, height and width, respectively.

Initializing

def __init__(self, size=2, stride=2, pad=0, useMask=False, name=None):

Parameters

Parameter Allowed types Description Default
size Union[int, tuple] Kernel size 2
stride Union[int, tuple] Pooling stride 2
pad Union[int, tuple] Input maps padding 0
useMask bool Whether to keep the tensor with maximum value indexes False
name str Layer name None

Explanations

size - possible to specify either a single size of the pooling kernel, in which case it will be square, or a tuple (size_h, size_w), where size_h is the height of the pooling kernel, and size_w is its width


stride - possible to specify either a single value of the pooling stride along all the axes of the map, or a tuple (stride_h, stride_w), where stride_h is the value of the pooling stride along the map height, and stride_w is along the width;


pad - possible to specify either a single padding value for all sides of the maps, or a tuple (pad_h, pad_w), where pad_h is the pooling value on each side along the height of the map, and pad_w is along the width. The possibility of creating an asymmetric padding (filling with additional elements on only one side of the tensor) is not provided for this module, please use Pad2D.

Examples


Basic pooling example


Necessary imports.

import numpy as np
from PuzzleLib.Backend import gpuarray
from PuzzleLib.Modules import MaxPool2D

Info

gpuarray is required to properly place the tensor in the GPU

Let us set the tensor parameters to clearly demonstrate the operation of the module.

batchsize, maps, h, w = 1, 1, 6, 6
data = gpuarray.to_gpu(np.arange(batchsize * maps * h * w).reshape((batchsize, maps, h, w)).astype(np.float32))
print(data)
[[[[ 0.  1.  2.  3.  4.  5.]
   [ 6.  7.  8.  9. 10. 11.]
   [12. 13. 14. 15. 16. 17.]
   [18. 19. 20. 21. 22. 23.]
   [24. 25. 26. 27. 28. 29.]
   [30. 31. 32. 33. 34. 35.]]]]

Let us initialize the module with standard parameters (size=2, stride=2, pad=0, useMask=False):

pool = MaxPool2D()
print(avgpool(data))
[[[[ 7.  9. 11.]
   [19. 21. 23.]
   [31. 33. 35.]]]]


Size parameter


Let us leave all parameters the same except for size:

pool = MaxPool2D(size=4)
print(pool(data))
[[[[21. 23.]
   [33. 35.]]]]

The size parameter can be set different for the height and width of the map:

pool = MaxPool2D(size=(4, 2))
print(pool(data))
[[[[19. 21. 23.]
   [31. 33. 35.]]]]


Stride parameter


Let us set stride value to 1:

pool = MaxPool2D(stride=1)
print(pool(data))
[[[[ 7.  8.  9. 10. 11.]
   [13. 14. 15. 16. 17.]
   [19. 20. 21. 22. 23.]
   [25. 26. 27. 28. 29.]
   [31. 32. 33. 34. 35.]]]]

The stride parameter can also be set different for the height and width of the map:

pool = MaxPool2D(stride=(1, 3))
print(pool(data))
[[[[ 7. 10.]
   [13. 16.]
   [19. 22.]
   [25. 28.]
   [31. 34.]]]]

Let us change both stride and size:

pool = MaxPool2D(size=4, stride=4)
print(pool(data))
[[[[21.]]]]

Only one element was formed in the output tensor, since the remaining elements of the input tensor could not form subtensors that were no smaller than the pooling window, which is why they were ignored.


Pad parameter


To include the ignored elements from the previous example, let us initialize the padding:

pool = MaxPool2D(size=4, stride=4, pad=1)
print(pool(data))
[[[[14. 17.]
   [32. 35.]]]]
Please note that padding in the module is always symmetric: one new element (row or column) was added on each side of the original tensor, i.e. after padding it would look as follows:
[[[[ 0.  0.  0.  0.  0.  0.  0.  0.]
   [ 0.  0.  1.  2.  3.  4.  5.  0.]
   [ 0.  6.  7.  8.  9. 10. 11.  0.]
   [ 0. 12. 13. 14. 15. 16. 17.  0.]
   [ 0. 18. 19. 20. 21. 22. 23.  0.]
   [ 0. 24. 25. 26. 27. 28. 29.  0.]
   [ 0. 30. 31. 32. 33. 34. 35.  0.]
   [ 0.  0.  0.  0.  0.  0.  0.  0.]]]]
The pad parameter can also be set different for the height and the width of the map. For example pad=(0, 2), then:
[[[[ 0.  0.  0.  1.  2.  3.  4.  5.  0.  0.]
   [ 0.  0.  6.  7.  8.  9. 10. 11.  0.  0.]
   [ 0.  0. 12. 13. 14. 15. 16. 17.  0.  0.]
   [ 0.  0. 18. 19. 20. 21. 22. 23.  0.  0.]
   [ 0.  0. 24. 25. 26. 27. 28. 29.  0.  0.]
   [ 0.  0. 30. 31. 32. 33. 34. 35.  0.  0.]]]]


useMask parameter


The useMask parameter is responsible for preserving the maximum elements indexes tensor. To demonstrate the way it operates, let us reinitialize the data tensor:

np.random.seed(123)
data = gpuarray.to_gpu(np.random.randint(low=0, high=9, size=(batchsize, maps, h, w)).astype(np.float32))
print(data)
[[[[2. 2. 6. 1. 3. 6.]
   [1. 0. 1. 0. 0. 3.]
   [4. 0. 0. 4. 1. 7.]
   [3. 2. 4. 7. 2. 4.]
   [8. 0. 7. 3. 4. 6.]
   [1. 5. 6. 2. 1. 8.]]]]
pool = MaxPool2D(useMask=True)
print(pool(data))
[[[[2. 6. 6.]
   [4. 7. 7.]
   [8. 7. 8.]]]]
print(pool.mask)
[[[[ 6  3 10]
   [19 20 17]
   [30 27 34]]]]

The indexes are returned separately for each batch element and each map:

maps = 2
data = gpuarray.to_gpu(np.random.randint(low=0, high=9, size=(batchsize, maps, h, w)).astype(np.float32))
print(data)
[[[[2. 2. 6. 1. 3. 6.]
   [1. 0. 1. 0. 0. 3.]
   [4. 0. 0. 4. 1. 7.]
   [3. 2. 4. 7. 2. 4.]
   [8. 0. 7. 3. 4. 6.]
   [1. 5. 6. 2. 1. 8.]]

  [[3. 5. 0. 2. 6. 2.]
   [4. 4. 6. 3. 0. 6.]
   [4. 7. 6. 7. 1. 5.]
   [7. 2. 4. 8. 1. 2.]
   [1. 1. 3. 5. 0. 8.]
   [1. 6. 3. 3. 5. 7.]]]]
pool = MaxPool2D(useMask=True)
print(pool(data))
[[[[2. 6. 6.]
   [4. 7. 7.]
   [8. 7. 8.]]

  [[5. 6. 6.]
   [7. 8. 5.]
   [6. 5. 8.]]]]
print(pool.mask)
[[[[ 0  2  5]
   [12 21 17]
   [24 26 35]]

  [[ 1  8  4]
   [13 21 17]
   [31 27 29]]]]
print(pool.mask.shape)
(1, 2, 3, 3)