Convolutional networks: convolution, pooling, receptive field

A convolutional layer slides small learned filters across an image, computing a dot product at every position, so the same pattern detector is reused everywhere. Stacking layers builds up from edges to whole objects.

Why it matters

A fully connected layer on a 224 × 224 colour image would need about 150,000 weights per neuron, and would have to learn "cat" separately in every corner. Convolutions share a few weights across all positions, which saves enormous numbers of parameters and builds in the fact that a pattern means the same thing wherever it appears.

The idea

A convolution slides a small filter, like 3×3, over the image and computes a dot product at each position. The same filter is used everywhere, so a cat is a cat in any corner. Pooling shrinks the image. The receptive field is how much of the original image one output can see, and it grows with depth.

Scanning a page with a magnifying glass for one pattern, like an edge. Deeper layers look for patterns made of patterns: edges, then corners, then eyes, then faces.

A magnifying glass is one tool. A CNN layer has many filters at once, and they are learned from data, not designed by hand.

How it works

  1. A filter (kernel) is a small grid of weights, for example 3 × 3.
  2. Place it on the top-left patch of the image, multiply matching numbers and add them up: one output value.
  3. Slide it across and down (by the stride) to fill a feature map. Big values mean "this pattern is here".
  4. A layer has many filters, each producing its own feature map, so the output has many channels.
  5. Output size = (W − K + 2P)/S + 1, for an input W wide, a filter K wide, padding P and stride S. With padding P = 1, a 3 × 3 filter at stride 1 keeps the size the same.
  6. Pooling (for example 2 × 2 max pooling) shrinks the maps by keeping the strongest response in each patch.
  7. Deeper layers combine earlier maps, and each output sees a bigger patch of the original image: its receptive field grows from 3 × 3 to 5 × 5 to 7 × 7 …

Key points

Common mistakes

Not true: Without padding, a 3 × 3 convolution keeps the feature map the same size.
In fact: Without padding the map shrinks by 2 in each direction at every layer. A padding of 1 keeps the size for a 3 × 3 filter.

Not true: A convolutional layer needs more parameters when the image is larger.
In fact: The same small filters slide over the whole image, so the count is K × K × C_in × C_out + C_out, whatever the image size.

Part of Unit 3: Deep learning foundations in the free ML Foundations course.