Caching

A cache keeps a copy of frequently used data in fast storage, usually memory, so it can be served without repeating slow work.

Why it matters

Reading from a database on disk, or recalculating a result, is slow and expensive compared with reading memory. When the same data is read again and again, repeating the work is waste.

The idea

Caches exist at many layers: the browser, the CDN, the application (Redis or Memcached) and the database. The usual pattern is cache-aside: read from the cache; on a miss, read the database and store the result with a TTL. The hard parts are keeping the cache in step with the database (invalidation) and choosing what to remove when it's full (eviction, often LRU).

Keeping the books you use every day on your desk instead of walking to the library each time.

Books on your desk don't change by themselves. Data does: when the database changes, your cached copy is stale until it's updated or expires. That's the hardest problem in caching.

How it works

  1. A request needs product 42. The app asks Redis for the key 'product:42'.
  2. Hit: Redis has it and returns it in under a millisecond.
  3. Miss: the app reads the database (perhaps 5–20 ms), returns the result, and stores it in Redis with a TTL of, say, 5 minutes.
  4. When product 42 changes, the app deletes 'product:42' from Redis, so the next read loads the fresh value.
  5. When Redis is full, it evicts keys, usually the least recently used ones.
  6. Measure the hit rate: 95% means only 1 read in 20 reaches the database.

Example

A product page is read 1,000 times for every update. Cache-aside in Redis with a 5-minute TTL serves almost every read from memory, and the database sees a tiny share of the traffic.

Key points

Common mistakes

Not true: Important data: it can live in the cache alone, with no copy in a database.
In fact: A cache can be emptied at any moment: on a restart, or when memory runs out. The database stays the source of truth.

Not true: Cache-aside: the cache is filled when the data is written.
In fact: Cache-aside fills the cache on a read miss: load from the database, then store the value. Filling the cache on every write is write-through.

In the interview

Reads far outnumber writes here, so I'd add a Redis cache using cache-aside with a TTL. On writes I'd delete the cache key so the next read loads fresh data. I'd watch the hit rate, and protect hot keys from stampedes by letting one request rebuild while others wait.

Part of Unit 2: Core concepts: scale, reliability and speed in the free System Design course.