Design a payment system

Process payments reliably: charge customers, record every money movement, and never lose money or charge twice.

Why interviewers ask it

It tests correctness under failure — retries, timeouts, idempotency, transactions and audit trails — where a small bug means real money.

Functional requirements

Non-functional requirements

From version 1 to the final design

  1. Version 1: call the payment provider and mark the order 'paid'.
  2. Version 2: a payments table with statuses and idempotency keys.
  3. Version 3: a double-entry ledger, written in the same transaction as the status change.
  4. Version 4: webhooks, an outbox, reconciliation, and handling results that are unknown.

The design, step by step

  1. The payment service receives the request with an idempotency key; a unique constraint in the database blocks duplicates.
  2. It creates a PENDING payment record, then calls the payment provider. The provider handles the card details, so you never store raw card numbers.
  3. The provider's result (response or webhook) updates the status, and balanced double-entry ledger rows are written in the same transaction.
  4. The order service is told through an outbox event.
  5. A nightly reconciliation job compares the ledger with the provider's settlement files.

Deep dives

Common mistakes

Not true: When a charge times out, the safe move is to send a brand-new charge request.
In fact: The first charge may have gone through, so a new request could charge the customer twice. Retry with the same idempotency key, or ask the provider for the status.

In the interview

Correctness first. Every payment request carries an idempotency key enforced by a unique constraint. The payment service records a pending payment, then calls the payment provider. The provider holds the card data, so we never store card numbers. On the result, the service writes balanced double-entry ledger rows in one ACID transaction. Events go out through an outbox, and daily reconciliation against the provider catches any mismatch.

Part of Unit 8: High-level design questions in the free System Design course.