Unit 5: AI engineering
Turn foundation models into real products: retrieval, evaluation, agents and deployment. This is the core of most AI engineer jobs.
The big picture
Units 1 to 4 teach you how the engine works. Unit 5 teaches you to build the car around it: steering (prompts), fuel (data and retrieval), a dashboard (evaluation) and brakes (safety).
Lessons in this unit
- Prompting: clear instructions, examples, structured output: Prompting is writing the instructions, context and examples an LLM receives on each call; a good prompt is a complete, specific briefing that leaves the model little to guess.
- Structured outputs and function calling: Structured output makes the model reply in a fixed format that code can read, usually JSON that matches a schema. Function calling lets the model ask your code to run one of your functions, with the arguments it chose.
- Embeddings and semantic search: Semantic search finds texts with similar meaning by turning texts into embedding vectors and returning the ones whose vectors are closest to the query's vector.
- Vector databases and approximate nearest-neighbour search: A vector database stores embeddings with their metadata and an approximate nearest-neighbour (ANN) index, such as HNSW or IVF, that finds almost-the-closest vectors without comparing the query with every vector.
- RAG: chunking, retrieval and re-ranking: Retrieval-augmented generation (RAG) answers questions by retrieving relevant pieces of your documents and giving them to the model in the prompt, so it answers from that evidence.
- Context engineering: long context and memory: Context engineering is choosing exactly what goes into the model's context window on each call: instructions, examples, retrieved documents, history and tool results. The aim is to give it what it needs and nothing that distracts it.
- Evaluation: test sets, LLM-as-judge, error analysis: Evaluation measures whether an LLM system does its job. It uses a fixed test set of real inputs, automatic checks or judges for scoring, and careful reading of the failures to find what to fix.
- Agents and tool use: An agent is an LLM running in a loop. It decides an action (usually a tool call), sees the result, and decides again, until the task is done or a limit is reached.
- MCP: connecting models to tools and data: The Model Context Protocol (MCP) is an open standard that lets any AI application connect to tools and data through MCP servers. Each integration is built once, and works with every MCP-compatible app.
- Choosing between prompting, RAG and fine-tuning: Prompting, RAG and fine-tuning are three ways to adapt an LLM: prompting changes the instructions, RAG supplies knowledge at question time, and fine-tuning changes the model's weights to change its behaviour.
- Dataset engineering for fine-tuning: Dataset engineering is designing, collecting, cleaning and checking the examples you fine-tune on, because a fine-tuned model learns exactly what its data shows, mistakes included.
- Serving: APIs, Docker, latency and cost: Serving puts a model behind an API so applications can use it, packaged (often in a Docker container) with everything it needs, and measured for latency, throughput and cost.
- Caching, model routing and cost control: Cost control for LLM apps uses caching (reuse answers or processed prompts), routing (send easy requests to cheaper models) and budgets (limits, timeouts and per-request cost tracking).
- Observability: tracing and debugging LLM apps: Observability means recording what your LLM app did on every request: prompts, retrieved chunks, tool calls, outputs, tokens, time and cost. The record is a trace you can inspect, search and learn from.
- Safety: guardrails and prompt injection: Prompt injection is text hidden inside data the model reads (a web page, email or document) that tries to override its instructions. Guardrails are the checks, permission limits and approvals that limit the damage.
- ML system design: pipelines, training vs serving, drift: ML system design is building everything around a model — data collection, labelling, features, training, serving, monitoring and retraining — so it keeps working in production as the world changes.