Vertical vs Horizontal Scaling Explained

Scalability is the ability to handle more work by adding resources. There are two basic ways.

Vertical and horizontal#

Vertical:    [ small ]  -->  [   BIG   ]

Horizontal:  [ small ]  -->  [ small ] [ small ] [ small ]

Vertical scaling is the simple first step, and it goes further than many people expect. Horizontal scaling is how very large systems grow, and it also gives you redundancy.

From one server to many#

A typical path looks like this.

Stage 1: one server. App and database share a machine.

Stage 2: separate the database. App and database no longer compete for memory and CPU, and each can scale on its own.

Stage 3: several app servers behind a load balancer. This is the first horizontal step.

Stage 4: a cache and a CDN. These take load off the database and app servers.

Stage 5: read replicas. They handle more database reads.

Stage 6: queues. Slow work moves out of the request.

Stage 7: sharding. Data is split when writes outgrow one database.

Do not jump to stage 7. Each stage adds cost and complexity, so add it when you need it.

Stateless servers#

To scale horizontally, any server must be able to handle any request. That means servers should be stateless: they keep no user data in their own memory.

Stateful (bad for scaling):
  User logs in on Server A. Session is in A's memory.
  Next request goes to Server B. User appears logged out.

Stateless (good):
  Session is in shared storage, such as Redis.
  Any server can read it.

Likewise, uploaded files belong in object storage, not on an app server's disk. Stateless servers can be added, removed or replaced at any time, which also makes deployment easy.

Where is the bottleneck?#

Scaling only helps where the limit is. Adding app servers does nothing if the database is the slow part. Think of a request as a pipeline: it can go only as fast as the narrowest point.

capacity = {"Load balancer": 100_000, "App servers": 20_000, "Cache": 80_000, "Database": 5_000}
limit = min(capacity, key=capacity.get)
print(f"System limit: {capacity[limit]:,} requests/s, set by the {limit}")

capacity["App servers"] *= 2
print(f"After doubling app servers: {min(capacity.values()):,} requests/s")

capacity["Database"] *= 10
limit = min(capacity, key=capacity.get)
print(f"After scaling the database: {capacity[limit]:,} requests/s, now set by the {limit}")

System limit: 5,000 requests/s, set by the Database
After doubling app servers: 5,000 requests/s
After scaling the database: 40,000 requests/s, now set by the App servers

Measure first, using monitoring, then fix the real limit. Then the bottleneck moves somewhere else, and you repeat.

Auto-scaling#

Cloud platforms can add and remove servers automatically, usually based on CPU or request rate. You pay for what you use, but new servers take time to start, so keep some spare capacity for sudden spikes.

Common mistakes#

Practice#

app_limit = 4 * 1500
db_limit = 4000 / 2
print(f"App servers: {app_limit:,} requests/s")
print(f"Database: {db_limit:,.0f} requests/s")
print(f"System limit: {min(app_limit, db_limit):,.0f} requests/s")

App servers: 6,000 requests/s
Database: 2,000 requests/s
System limit: 2,000 requests/s

Key takeaway#

Next: Availability and Reliability.