System Design course menu
Advanced System Design Interview Questions
Advanced system design questions with answers: distributed transactions, sagas and the outbox pattern, consensus, unique IDs, multi-region design, hot keys and global rate limiting.
On this page
- 1. Two-phase commit or saga?
- 2. What is the outbox pattern?
- 3. What is change data capture?
- 4. How do systems such as Raft reach consensus?
- 5. How do you avoid split brain?
- 6. How do you generate unique IDs across many servers?
- 7. Why are clocks a problem in distributed systems?
- 8. How do you design for multiple regions?
- 9. How do you handle a hot key?
- 10. How do you count billions of events cheaply?
- 11. What is a Bloom filter?
- 12. How do LSM trees differ from B-trees?
- 13. How do you migrate a live database with no downtime?
- 14. How do you change a schema safely?
- 15. What is backpressure?
- 16. What is load shedding?
- 17. How do you prevent a retry storm?
- 18. How would you test a system's reliability?
- 19. How do you do safe deployments?
- 20. How do you reason about cost?
Senior interviews focus on failure, consistency across services, and running globally.
1. Two-phase commit or saga?#
Show solution
Two-phase commit coordinates an atomic commit across systems, but it blocks if the coordinator fails, and it scales poorly. A saga is a sequence of local transactions, with compensating steps. It scales better, but the data is only eventually consistent.
2. What is the outbox pattern?#
Show solution
To avoid a database write succeeding while an event publish fails, you write the event to an outbox table in the same transaction. A separate process then publishes it, reading the table or using change data capture.
3. What is change data capture?#
Show solution
Reading a database's change log, and streaming the changes to other systems, such as search or caches. Debezium is a common tool.
4. How do systems such as Raft reach consensus?#
Show solution
Nodes elect a leader by majority vote. The leader replicates a log, and an entry is committed once a majority stores it. A majority always overlaps, so committed data survives a minority of failures.
5. How do you avoid split brain?#
Show solution
Require a majority (quorum) to act as leader, and use fencing tokens, so that an old leader's writes are rejected.
6. How do you generate unique IDs across many servers?#
Show solution
UUIDs need no coordination, but are long and random, which suits indexes less well. Snowflake-style IDs combine time, machine ID and a sequence, in 64 bits, and they sort by time. Newer UUIDv7 values are also time-ordered.
7. Why are clocks a problem in distributed systems?#
Show solution
Machines' clocks drift, so timestamps cannot reliably order events. Use logical clocks or sequence numbers, or specialised time services, such as Spanner's TrueTime.
8. How do you design for multiple regions?#
Show solution
Decide between active-passive (simpler, with failover) and active-active (lower latency, but conflicts). Partition users by home region, replicate asynchronously, and plan for data residency laws.
9. How do you handle a hot key?#
Show solution
Cache it locally, replicate it, or split it into several keys and sum them, as with a counter. Add a random suffix for writes, to spread the load.
10. How do you count billions of events cheaply?#
Show solution
Aggregate in a stream with periodic flushes. For approximate counts of unique items, use HyperLogLog, and for frequency counts, use a Count-Min Sketch.
11. What is a Bloom filter?#
Show solution
A small structure that says an item is possibly present, or definitely absent. It avoids unnecessary disk reads and cache-penetration lookups, with a few false positives but no false negatives.
12. How do LSM trees differ from B-trees?#
Show solution
B-trees update pages in place, and they are good for reads. LSM trees buffer writes in memory, and flush sorted files, which suits heavy writes, but reads and compaction cost more. RocksDB and Cassandra use them.
13. How do you migrate a live database with no downtime?#
Show solution
Write to both old and new (dual writes), backfill old data, compare, then move reads gradually, and finally stop writing to the old one. Keep a rollback path.
14. How do you change a schema safely?#
Show solution
Use expand and contract. Add the new column, deploy code that handles both, backfill, switch reads, and remove the old column later.
15. What is backpressure?#
Show solution
A way for a slow consumer to signal producers to slow down, through bounded queues, rejected work or flow control, rather than failing from overload.
16. What is load shedding?#
Show solution
Deliberately rejecting some work when overloaded, starting with the least important, so that the system keeps serving the rest.
17. How do you prevent a retry storm?#
Show solution
Use exponential backoff with jitter, retry budgets, circuit breakers, and retry only at one layer.
18. How would you test a system's reliability?#
Show solution
Use load tests, game days, and chaos engineering, which injects failures in a controlled way, along with regular failover and restore tests.
19. How do you do safe deployments?#
Show solution
Use canary or blue-green releases, feature flags, automatic rollback on bad metrics, and backward-compatible changes.
20. How do you reason about cost?#
Show solution
Find the largest drivers, usually storage, network traffic and compute. Use caching, compression, cold storage and autoscaling, and estimate the cost per user.
Finished this lesson?
Related lessons
Recommended System Design resources
- System Design Interview by Alex Xu
Step-by-step walkthroughs of common design questions, for interview preparation.
Some links are affiliate links. If you buy through them, Laviyx Learn may earn a commission at no extra cost to you. As an Amazon Associate, we earn from qualifying purchases.