Unit 9: AI and ML system design
The design questions AI and ML engineer interviews ask: RAG assistants, recommendations, fraud detection, LLM serving, and the pipeline that trains and ships models.
The big picture
ML system design is normal system design plus three extra worries: where the data and features come from, how you know the model is good (evaluation), and what happens when the world changes (drift). Use the same framework, and give those three their own deep dives.
Lessons in this unit
- Design a RAG assistant over company documents: A chat assistant that answers employees' questions from the company's own documents, with sources.
- Design a recommendation system: Recommend items (videos, products, posts) that each user is likely to engage with.
- Design a real-time fraud detection system: Score every card transaction for fraud within milliseconds, and block or flag the risky ones.
- Design an LLM serving platform: Serve large language models to many applications with low latency and cost.
- Design an ML training and feature pipeline: The platform that turns raw data into trained, deployed and monitored models, again and again.