PyTorch: nn.Module, DataLoader and the training loop

In PyTorch, an nn.Module holds the layers and weights, a DataLoader serves shuffled mini-batches, and the training loop repeats the same few steps: forward, loss, zero_grad, backward, step.

Why it matters

Every deep-learning project, from MNIST to fine-tuning an LLM, uses this same skeleton. Getting its details wrong — no zero_grad, no eval mode, the wrong device — causes silent bugs that waste days. Knowing the loop cold lets you read any training script and write your own quickly.

The idea

nn.Module holds your layers and weights. DataLoader serves shuffled mini-batches. The training loop is almost always the same five steps.

A kitchen routine: get ingredients (a batch), cook (forward), taste (loss), adjust the recipe (backward and step), and clean the pan (zero_grad).

If you forget to clean the pan, flavours mix. If you forget zero_grad(), gradients from old batches add up.

How it works

  1. Define the model as a class that inherits nn.Module: create the layers in __init__ and use them in forward. Its parameters are registered automatically.
  2. Wrap your data in a Dataset (how to get item i) and a DataLoader (batch size, shuffling, worker processes).
  3. Create an optimiser over model.parameters(), for example torch.optim.AdamW(model.parameters(), lr=3e-4).
  4. For each batch: out = model(xb), then loss = loss_fn(out, yb).
  5. opt.zero_grad() clears old gradients, loss.backward() computes new ones, and opt.step() updates the weights.
  6. Validation: call model.eval() and use with torch.no_grad(): to measure without dropout or gradient tracking. Then call model.train() again.
  7. Move the model and every batch to the same device: model.to(device), xb.to(device).

Key points

Common mistakes

Not true: A model on the GPU can read a batch that is still on the CPU; it just runs more slowly.
In fact: PyTorch raises a device-mismatch error: a model and its inputs must be on the same device. Move each batch with .to(device).

Not true: Adding up loss tensors over an epoch, without .item(), costs nothing extra.
In fact: Each loss tensor keeps its whole graph alive, so memory grows all epoch. Add loss.item(), a plain number, instead.

Part of Unit 3: Deep learning foundations in the free ML Foundations course.