Scalability: vertical vs horizontal
Scalability is a system's ability to handle more load by adding resources.
Why it matters
A system that works for 100 users can fail at 100,000. You need a way to add capacity without rewriting everything.
The idea
Vertical scaling means a bigger machine (more CPU and memory). It's simple, but there's a ceiling, and that one machine can fail. Horizontal scaling means more machines behind a load balancer. It has no hard ceiling and survives failures, but needs stateless app servers and shared storage. The key trick: keep app servers stateless, and keep sessions and data in shared stores.
A busy restaurant can buy a bigger oven (vertical) or open more kitchens (horizontal).
More kitchens only help if any kitchen can cook any order. If half of a customer's order is in kitchen 2, it has to go back there. That's 'state', and it's why app servers should be stateless.
How it works
- Find the bottleneck first: is the CPU at 100%, the database slow, memory full, or the network saturated?
- Make app servers stateless: no sessions or files kept on the server; put them in Redis and object storage instead.
- Put a load balancer in front and add servers. With auto-scaling, servers are added automatically when CPU use or latency rises.
- Take reads off the database: cache hot data and add read replicas.
- When writes become the bottleneck, shard the database by a key.
- Move slow work off the request path with queues.
Example
Moving login sessions from server memory into Redis lets any server handle any user, so you can add servers freely.
Key points
- Stateless app servers plus a shared session store = easy horizontal scaling.
- Databases are the hard part: read replicas for more reads, sharding for more writes.
- Scale the real bottleneck: measure first (CPU, memory, database, network).
- Auto-scaling adds and removes servers based on load.
- Scalability is not speed: a system fast for one user can still collapse under many.
Trade-offs
- Vertical: simple, no code changes, but a ceiling and a single point of failure.
- Horizontal: almost unlimited and fault tolerant, but needs statelessness, load balancing and distributed data.
Common mistakes
- Adding app servers when the database is the real bottleneck.
- Keeping sessions in server memory, then adding more servers.
- Sharding far too early: it adds a lot of complexity.
Not true: A slow system: more app servers make it faster, even when the database is the bottleneck.
In fact: Only when the app servers are the bottleneck. If the database is the slow part, more app servers just send it more work, so measure first.
Not true: Sessions in a server's memory: they keep working when more servers are added.
In fact: The next request may reach a different server, which knows nothing about the session. Keep sessions in a shared store, so the app servers stay stateless.
In the interview
I'd keep the application tier stateless so it scales horizontally behind a load balancer with auto-scaling. State goes into shared stores: sessions in Redis, data in the database. For the database, I'd add read replicas first and shard only when write volume needs it.
Part of Unit 2: Core concepts: scale, reliability and speed in the free System Design course.