Understanding Stride in Convolutional Neural Networks

In the context of convolutional neural networks (CNNs), stride refers to the number of pixels by which the filter or kernel moves across the input image during the convolution operation. This parameter plays a crucial role in determining the spatial dimensions of the feature maps and the overall computational complexity of the network.

Key Points about Stride

Stride Value

Stride of 1: When the stride is set to 1, the filter moves one pixel at a time, covering every pixel in the input image. This results in an output feature map of the same size as the input.

Stride of 2: A stride of 2 means the filter moves two pixels at a time, effectively skipping every other pixel. This results in a smaller output feature map.

Impact on Output Size

The choice of stride significantly affects the spatial dimensions of the output feature map. Generally, larger strides produce smaller output sizes. The size of the output can be calculated using the following formula:

Output Size ?(Input Size - Filter Size)/Stride - 1?

Here, Input Size is the dimension of the input, Filter Size is the dimension of the kernel, Padding refers to additional pixels added around the input, and Stride is the step size.

Affects on Learning

Reduced Computational Load: Using a larger stride can reduce the computational load and memory requirements by decreasing the size of the output feature map. However, this can also lead to a loss of spatial information and details.

Common Practices: Stride values are typically set to 1 or 2 in many CNN architectures depending on the desired level of downsampling and the specific application. A stride of 1 is often used for models that need to maintain spatial resolution, while a stride of 2 is commonly used for downsampling.

Example

Consider the following example:

Shown above is an image of size 4x4, which will be the original image on which the convolution operation will be performed. A simple 2x2 filter is used for the convolution. As the filter moves across the image, it covers different segments of the image, effectively skipping every other pixel. This can be visualized in the following steps:

The filter starts at the top-left corner and covers the first 2x2 block. Then, the filter moves right by one pixel, covering the next 2x2 block. Next, the filter moves down by one pixel and left by one pixel to cover the next 2x2 block. This process continues until the entire image has been covered.

After the convolution is complete, the result is a 3x3 feature map, as shown below.

Conclusions

The stride parameter significantly influences the size and spatial resolution of the output in CNNs. When the filter is moved across the entire image, the output can be of a different size, the same size, or a larger size based on other parameters such as 'padding.'

An important consideration is that higher stride results in a lower dimension of the output image, which can both save computational resources and reduce the spatial resolution of the feature maps.