Fine-tuning and LoRA

Fine-tuning continues training a pretrained model on your own data. LoRA does it cheaply: it freezes the original weights W and trains a small low-rank update, so the layer uses W + B·A.

Why it matters

Pretrained models are general; you often need a specific style, format or domain. Full fine-tuning updates billions of weights, needs optimiser memory for all of them, and produces a whole new model copy per task. LoRA trains well under 1% of the parameters, fits on a single GPU, and produces small adapters you can swap in and out of one base model.

The idea

Fine-tuning continues training a pretrained model on your own data. Full fine-tuning updates every weight. LoRA freezes the original weights W and learns a small update BA of low rank r (for example 8): W′ = W + BA.

Instead of rewriting a whole textbook, you add a few sticky notes with corrections.

Sticky notes can be removed, and so can LoRA adapters, so one base model can serve many tasks. But a small rank cannot teach big new skills.

How it works

  1. Pick the weight matrices to adapt, usually the attention projections (and often the MLP).
  2. Freeze W. Add two small matrices: B (d_out × r) and A (r × d_in), with rank r such as 8 or 16.
  3. Start B at zero, so B·A = 0 and the model begins exactly as the pretrained one.
  4. The layer computes y = x·Wᵀ + x·(B·A)ᵀ, usually scaled by α/r.
  5. Train only A and B. For a 4,096 × 4,096 matrix with r = 8, that is 65,536 weights instead of 16.8 million: about 0.4%.
  6. Afterwards, keep the adapter separate (tiny files, swappable), or merge it: W′ = W + B·A, with no extra cost when running.
  7. QLoRA goes further: the frozen base model is stored in 4-bit, so even large models can be fine-tuned on one GPU.

Key points

Common mistakes

Not true: Facts that change each week: add them by fine-tuning.
In fact: Retraining for each change is slow and costly, and the old facts stay in the weights. Facts that change belong in retrieval (RAG), not in fine-tuning.

Not true: LoRA rank: r = 1 covers a big change in behaviour as well as a small one.
In fact: The rank limits how much the update can express. A big change in behaviour needs a higher rank; a small dataset needs a lower one, or it overfits.

Part of Unit 4: Transformers and LLMs in the free ML Foundations course.