Unique ID generation
Generating IDs that are unique across many servers, without one shared database counter.
Why it matters
Every order, message and user needs an ID that never repeats. With many servers creating records at the same moment, one shared counter becomes a bottleneck and a single point of failure.
The idea
A single auto-increment counter becomes a bottleneck at scale. There are three options. UUIDs: random, no coordination, 128-bit, but version 4 isn't ordered by time. A ticket server that hands out ranges of numbers. Or Snowflake-style IDs: 64-bit numbers made from a timestamp, a machine ID and a sequence number. They're unique, roughly ordered by time, and generated locally.
Ticket numbers at a deli with several ticket machines. Each machine prints its own machine code plus the time, so tickets never clash.
Deli machines share one wall clock. Servers' clocks drift, and a clock that jumps backwards could create duplicate Snowflake IDs, so the generator must notice and wait.
How it works
- A Snowflake ID is built as: timestamp (41 bits), then machine ID (10 bits), then sequence (12 bits).
- Each machine gets a unique machine ID when it starts, from configuration or a coordinator.
- For each new ID within the same millisecond, the machine adds one to its sequence (0 up to 4,095).
- If the sequence runs out, the machine waits for the next millisecond.
- Because the timestamp comes first, IDs made later are bigger, so they sort roughly by time.
Example
Snowflake: 41 bits of milliseconds since a chosen start date, 10 bits of machine ID, and 12 bits of sequence, so each machine can make 4,096 IDs per millisecond.
Key points
- Auto-increment: simple, but a single point of failure and a bottleneck, and it reveals your counts.
- UUID v4: no coordination, 128 bits, random order (bad for inserts into a B-tree index).
- UUID v7 and ULID: ordered by time and still need no coordination.
- Snowflake: 64 bits, sortable by time; needs unique machine IDs and sane clocks.
- Range allocation: each server reserves a block of, say, 1,000 IDs at a time.
Common mistakes
- Two machines accidentally sharing a machine ID.
- Ignoring clock problems: time-based IDs can repeat or go backwards when a server's clock is corrected.
- Exposing sequential IDs that let outsiders count your orders or guess other users' links.
Not true: Time-based IDs: no repeats, because a server's clock moves forward.
In fact: A server's clock can be corrected backwards, and two machines can make IDs in the same millisecond. Time-based IDs also need a machine ID and a sequence number.
Not true: Sequential IDs in public links: they reveal no information to outsiders.
In fact: With IDs 1001, 1002, 1003, an outsider can count your orders and guess other people's links. Use random-looking IDs in anything public.
In the interview
I'd use Snowflake-style 64-bit IDs: timestamp, machine ID and a per-millisecond sequence. They're generated locally without coordination, they're unique, and they sort roughly by time, which keeps database indexes efficient.
Part of Unit 3: Data: databases, storage and scaling data in the free System Design course.