PyTorch: nn.Module, DataLoader and the training loop
In PyTorch, an nn.Module holds the layers and weights, a DataLoader serves shuffled mini-batches, and the training loop repeats the same few steps: forward, loss, zero_grad, backward, step.
Why it matters
Every deep-learning project, from MNIST to fine-tuning an LLM, uses this same skeleton. Getting its details wrong — no zero_grad, no eval mode, the wrong device — causes silent bugs that waste days. Knowing the loop cold lets you read any training script and write your own quickly.
The idea
nn.Module holds your layers and weights. DataLoader serves shuffled mini-batches. The training loop is almost always the same five steps.
A kitchen routine: get ingredients (a batch), cook (forward), taste (loss), adjust the recipe (backward and step), and clean the pan (zero_grad).
If you forget to clean the pan, flavours mix. If you forget zero_grad(), gradients from old batches add up.
How it works
- Define the model as a class that inherits nn.Module: create the layers in __init__ and use them in forward. Its parameters are registered automatically.
- Wrap your data in a Dataset (how to get item i) and a DataLoader (batch size, shuffling, worker processes).
- Create an optimiser over model.parameters(), for example torch.optim.AdamW(model.parameters(), lr=3e-4).
- For each batch: out = model(xb), then loss = loss_fn(out, yb).
- opt.zero_grad() clears old gradients, loss.backward() computes new ones, and opt.step() updates the weights.
- Validation: call model.eval() and use with torch.no_grad(): to measure without dropout or gradient tracking. Then call model.train() again.
- Move the model and every batch to the same device: model.to(device), xb.to(device).
Key points
- The five lines: forward, loss, zero_grad, backward, step.
- model.train() and model.eval() switch dropout and batch norm behaviour.
- torch.no_grad() for evaluation: no graph, less memory.
- Shuffle the training loader, not the validation loader.
- Log the training and validation loss every epoch, and save checkpoints (the model and optimiser state).
- Set random seeds to make runs reproducible.
Common mistakes
- Forgetting opt.zero_grad(), so the gradients of earlier steps are added to the new ones.
- Forgetting model.eval() before validation, so dropout and batch norm still behave as they do in training.
- Calling softmax before F.cross_entropy, which already applies it: the loss is then computed on the wrong numbers.
- Model on the GPU, data on the CPU (or the other way round): a device-mismatch error.
- Accumulating loss in a Python number without .item(), which keeps every graph alive and leaks memory.
Not true: A model on the GPU can read a batch that is still on the CPU; it just runs more slowly.
In fact: PyTorch raises a device-mismatch error: a model and its inputs must be on the same device. Move each batch with .to(device).
Not true: Adding up loss tensors over an epoch, without .item(), costs nothing extra.
In fact: Each loss tensor keeps its whole graph alive, so memory grows all epoch. Add loss.item(), a plain number, instead.
Part of Unit 3: Deep learning foundations in the free ML Foundations course.