Design a recommendation system
Recommend items (videos, products, posts) that each user is likely to engage with.
Why interviewers ask it
It tests the two-stage pattern (retrieve, then rank), feature consistency, feedback loops, and how to measure success.
Functional requirements
- Personalised home-page recommendations.
- 'Similar items' on each item's page.
- Learn from clicks, watches and purchases.
Non-functional requirements
- Results in about 100–200 ms.
- Fresh: reacts quickly to new behaviour.
- Millions of users and items.
From version 1 to the final design
- Version 1: show everyone the most popular items.
- Version 2: item-to-item similarity ('people who watched X also watched Y').
- Version 3: embeddings to gather candidates, plus a ranking model.
- Version 4: a feature store, re-ranking rules, exploration and A/B testing.
The design, step by step
- Two stages. Candidate generation retrieves about 500–1,000 items quickly (embedding similarity with approximate nearest-neighbour search, items often bought together, popular items). Ranking then scores those candidates with a richer model using user, item and context features.
- Re-ranking applies business rules: variety, freshness, removing items already seen.
- Models train offline on logged interactions; features come from a feature store, defined the same way for training and serving.
- Online A/B tests measure the real impact of every change.
Deep dives
- Cold start: new users get popular and contextual items; new items use content features.
- Feedback loops and popularity bias: add some exploration.
- Training-serving skew: one feature definition used both offline and online.
- Metrics: offline (recall@k, NDCG) and online (click-through, watch time, retention).
Common mistakes
- Scoring every item with a heavy model on every request.
- Judging success only by offline metrics.
- Ignoring the cold-start problem: new users and new items have no history, so they need a fallback such as popular items.
Not true: Offline metrics: a better score means users will like the recommendations more.
In fact: Offline scores are measured on past behaviour, which the old system shaped. Only an online A/B test shows what real users do.
In the interview
It is a two-stage system. Candidate generation pulls about a thousand items cheaply, using embedding similarity and simple sources. Then a ranking model scores them with user, item and context features from a feature store. A final re-ranking step applies variety and business rules. Models train offline on interaction logs, and every change is checked with an online A/B test.
Part of Unit 9: AI and ML system design in the free System Design course.