Unit 4: Transformers and LLMs

Build a small GPT from scratch: tokenizer, attention, training and fine-tuning. This is how modern AI works inside.

The big picture

An LLM is a deep neural network with one special layer (attention), trained to guess the next token on a huge amount of text. Everything else is detail, but important detail.

Lessons in this unit