Back-of-the-Envelope Estimation for System Design

Before designing, you need to know roughly how big the problem is. Estimates tell you whether one database is enough, or whether you need fifty. Aim for the right order of magnitude, not an exact answer.

Numbers to remember#

Traffic: requests per second#

QPS means queries per second. Start from users:

daily_users = 10_000_000
actions_per_user = 5
seconds_per_day = 86_400

avg_qps = daily_users * actions_per_user / seconds_per_day
peak_qps = avg_qps * 3

print(f"Average: {avg_qps:,.0f} QPS")
print(f"Peak:    {peak_qps:,.0f} QPS")

Average: 579 QPS
Peak:    1,736 QPS

Traffic is not even through the day, so peak is often estimated at two to five times the average. Design for the peak.

Read and write ratio#

Most apps read far more than they write. A social app might have 100 reads for each write, and that changes the design: lots of reads suggests caches and replicas.

writes_per_day = 2_000_000
read_ratio = 100

write_qps = writes_per_day / seconds_per_day
read_qps = write_qps * read_ratio
print(f"Writes: {write_qps:,.0f} per second")
print(f"Reads:  {read_qps:,.0f} per second")

Writes: 23 per second
Reads:  2,315 per second

Storage#

Multiply the number of new items by their size, and by how long you keep them:

photos_per_day = 2_000_000
photo_mb = 2
years = 5

per_day_tb = photos_per_day * photo_mb / 1_000_000
total_pb = per_day_tb * 365 * years / 1000
with_copies = total_pb * 3

print(f"Per day:         {per_day_tb:,.0f} TB")
print(f"In {years} years:      {total_pb:,.1f} PB")
print(f"With 3 copies:   {with_copies:,.1f} PB")

Per day:         4 TB
In 5 years:      7.3 PB
With 3 copies:   21.9 PB

Remember to include replicas and backups.

Bandwidth#

read_qps_photos = 5_000
mb_per_read = 2
gbps = read_qps_photos * mb_per_read * 8 / 1000
print(f"Outgoing: {gbps:,.0f} Gbps")

Outgoing: 80 Gbps

That much traffic is a strong reason to use a CDN.

How many servers?#

If one app server handles about 1,000 requests a second (this depends greatly on the work done), then:

import math

peak = 50_000
per_server = 1_000
headroom = 0.7

servers = math.ceil(peak / (per_server * headroom))
print(f"{servers} servers, running at most 70% busy")

72 servers, running at most 70% busy

Keeping spare capacity means that traffic can spike, or a server can fail, without trouble.

Latency numbers#

These rough values help you reason about where time goes. Exact numbers vary by hardware, but the differences between them matter:

Three lessons come from this:

• Memory is thousands of times faster than disk, which is why caches work.
• Going across a network is far slower than local work, so avoid many small calls.
• Distance matters, which is why content is placed near users.

Tips#

• Round freely: 86,400 becomes 100,000.
• State your assumptions out loud.
• Write down units, and check them.
• Use estimates to make decisions, and then move on.

Common mistakes#

Practice#

users = 50_000_000
messages = users * 40
per_second = messages / 86_400
tb_per_year = messages * 200 * 365 / 1_000_000_000_000
print(f"{per_second:,.0f} messages per second")
print(f"{tb_per_year:,.0f} TB per year")

23,148 messages per second
146 TB per year

Key takeaway#

Next: Scaling Up and Out.