Design a real-time fraud detection system
Score every card transaction for fraud within milliseconds, and block or flag the risky ones.
Why interviewers ask it
It tests real-time ML: low-latency features, imbalanced data, thresholds based on costs, labels that arrive late, and drift.
Functional requirements
- Score every transaction before it's approved.
- Allow, block, or ask for extra verification (such as a one-time code), or send to manual review.
- Learn from confirmed fraud.
Non-functional requirements
- A decision in under about 100 ms, with very high availability (fail safely).
- Very imbalanced data: fraud is far below 1%.
- Explainable decisions for regulators and customers.
From version 1 to the final design
- Version 1: hand-written rules (for example, block payments over £5,000 made abroad).
- Version 2: a model trained offline and scored in a service.
- Version 3: streaming features (like transaction counts in the last 10 minutes) in a feature store.
- Version 4: rules plus model plus a review queue, thresholds tuned by cost, drift monitoring and regular retraining.
The design, step by step
- Transaction → scoring service: fetch real-time features (how many transactions in the last 10 minutes, distance from usual location, device) and historical features → rules engine plus an ML model (often gradient-boosted trees) → decision using thresholds.
- A streaming pipeline (Kafka plus a stream processor) keeps the windowed counts and sums up to date.
- Every decision and its features are logged; fraud labels arrive later (chargebacks) and are joined for retraining.
- A case-management tool for fraud analysts.
Deep dives
- Class imbalance: choose the precision–recall trade-off from the cost of fraud versus the cost of blocking good customers.
- Delayed, noisy labels, and fraudsters changing tactics (drift): monitor and retrain often.
- If the model service is down or slow, fall back to rules only.
- Point-in-time correct features, so no future information leaks into training.
Common mistakes
- Using accuracy as the metric: if 0.1% of payments are fraud, a model that flags nothing is 99.9% accurate and useless.
- Features calculated with information from the future.
- No fallback when the model service fails.
Not true: Accuracy is a good metric for a fraud detection model.
In fact: If 0.1% of payments are fraud, a model that flags nothing is 99.9% accurate and useless. Use precision and recall instead.
In the interview
Each transaction hits a low-latency scoring service. It pulls real-time features from a feature store, which a streaming pipeline keeps fresh. It runs rules plus a gradient-boosted model, and returns allow, verify or block within about 100 milliseconds. Thresholds come from the business cost of fraud versus false declines. Decisions are logged, chargeback labels are joined later, and the model is monitored for drift and retrained regularly.
Part of Unit 9: AI and ML system design in the free System Design course.