Availability: nines, SLAs and redundancy

Availability is the share of time a system works and responds correctly, usually written in 'nines', like 99.9%.

Why it matters

Machines, disks, networks and whole datacentres fail. If your system has only one copy of anything important, that one failure takes everything down.

The idea

99.9% ('three nines') allows about 8.8 hours of downtime a year; 99.99% about 53 minutes. You raise availability by removing single points of failure: run several copies across machines and zones, with health checks and automatic failover. An SLO is your internal target; an SLA is the promise to customers, often with refunds if you miss it.

A hospital with backup generators. When the mains power fails, a generator starts and the lights stay on.

A generator is one independent backup. Real systems also fail in shared ways — a bad deployment, a whole region going down, a shared database — so backups only help if they fail independently.

How it works

  1. Agree the target. 99.9% means about 43 minutes of downtime a month.
  2. List every component a request touches and ask: if this dies, what happens?
  3. Add redundancy: at least two copies of each component, in different availability zones.
  4. Add detection: health checks that notice a failure within seconds.
  5. Add automatic failover: the load balancer stops sending traffic to dead servers, and a database replica is promoted when the leader dies.
  6. Practise it: test failover regularly, because untested backups often don't work when you need them.

Example

Two parts in series, each 99.9% available, give about 99.8% overall (0.999 × 0.999). Two redundant copies in parallel, each 99%, give 99.99% (1 − 0.01²), as long as they fail independently.

Key points

Trade-offs

Common mistakes

Not true: Depending on a 99.9% database: the service can still promise 99.99%.
In fact: A service is at most as available as what it depends on. Availabilities in series multiply, so the result is below 99.9%.

Not true: Two copies on one rack: no single point of failure left.
In fact: The rack is still a single point of failure: one power or network fault takes both copies down. Put the copies in different racks or zones.

In the interview

I'd first agree a target, say 99.9%. Then I'd remove single points of failure: several app servers across availability zones, a replicated database with automatic failover, redundant load balancers and health checks. I'd also remember that every dependency in series lowers the overall availability.

Part of Unit 2: Core concepts: scale, reliability and speed in the free System Design course.