Convolutional networks: convolution, pooling, receptive field
A convolutional layer slides small learned filters across an image, computing a dot product at every position, so the same pattern detector is reused everywhere. Stacking layers builds up from edges to whole objects.
Why it matters
A fully connected layer on a 224 × 224 colour image would need about 150,000 weights per neuron, and would have to learn "cat" separately in every corner. Convolutions share a few weights across all positions, which saves enormous numbers of parameters and builds in the fact that a pattern means the same thing wherever it appears.
The idea
A convolution slides a small filter, like 3×3, over the image and computes a dot product at each position. The same filter is used everywhere, so a cat is a cat in any corner. Pooling shrinks the image. The receptive field is how much of the original image one output can see, and it grows with depth.
Scanning a page with a magnifying glass for one pattern, like an edge. Deeper layers look for patterns made of patterns: edges, then corners, then eyes, then faces.
A magnifying glass is one tool. A CNN layer has many filters at once, and they are learned from data, not designed by hand.
How it works
- A filter (kernel) is a small grid of weights, for example 3 × 3.
- Place it on the top-left patch of the image, multiply matching numbers and add them up: one output value.
- Slide it across and down (by the stride) to fill a feature map. Big values mean "this pattern is here".
- A layer has many filters, each producing its own feature map, so the output has many channels.
- Output size = (W − K + 2P)/S + 1, for an input W wide, a filter K wide, padding P and stride S. With padding P = 1, a 3 × 3 filter at stride 1 keeps the size the same.
- Pooling (for example 2 × 2 max pooling) shrinks the maps by keeping the strongest response in each patch.
- Deeper layers combine earlier maps, and each output sees a bigger patch of the original image: its receptive field grows from 3 × 3 to 5 × 5 to 7 × 7 …
Key points
- Weight sharing: the same filter at every position, so far fewer parameters.
- Shift equivariance: move the input and the feature map moves too.
- Parameters of a conv layer = K × K × C_in × C_out + C_out, whatever the image size.
- Output size = (W − K + 2P)/S + 1, for an input W wide, a filter K wide, padding P and stride S.
- Receptive field grows with depth, stride and pooling.
- Vision transformers (Unit 4) now rival CNNs, but convolutions remain everywhere in vision, audio and on-device models.
Common mistakes
- Getting the output size wrong, then hitting shape errors at the first linear layer.
- Confusing channels-first (N, C, H, W, PyTorch) with channels-last (N, H, W, C).
- Forgetting padding, so the feature maps shrink at every layer.
Not true: Without padding, a 3 × 3 convolution keeps the feature map the same size.
In fact: Without padding the map shrinks by 2 in each direction at every layer. A padding of 1 keeps the size for a 3 × 3 filter.
Not true: A convolutional layer needs more parameters when the image is larger.
In fact: The same small filters slide over the whole image, so the count is K × K × C_in × C_out + C_out, whatever the image size.
Part of Unit 3: Deep learning foundations in the free ML Foundations course.