System Design

System Design Framework

A structured approach to designing large-scale distributed systems. Apply these principles when architecting new services, reviewing designs, estimating capacity, or preparing for system design discussions.

Core Principle

Start with requirements, not solutions. Jumping to architecture before understanding constraints produces over- or under-engineered systems. Scalable systems are assembled from well-understood building blocks (load balancers, caches, queues, databases, CDNs) — the skill lies in choosing the right blocks, sizing them with estimates, and owning the tradeoffs each choice introduces.

Scoring

Goal: 10/10. Score a design by how many of the eight Quick Diagnostic rows it satisfies — score = round(passed / 8 × 10): 9-10 = all/nearly all rows pass — explicit requirements, real estimates, redundancy, a stated DB-scaling and caching strategy, async via queues, monitoring, and a deployment plan, with tradeoffs named; 5-6 = the design works but skips estimation, redundancy, or operations; <=3 = architecture proposed before requirements or estimates exist. Always state the current score, name the failing diagnostic rows, and give the specific fix for each.

The System Design Framework

Six areas for building reliable, scalable distributed systems:

1. The Four-Step Process

Core concept: Every design follows four stages: (1) understand the problem and establish scope, (2) propose a high-level design and get buy-in, (3) dive deep into critical components, (4) wrap up with tradeoffs and future improvements.

Why it works: Without structure, designs either stay too abstract or get lost in premature detail. The four steps invest time proportionally — broad strokes first, depth where it matters.

Key insights:

• Step 1 (~5-10 min): clarifying questions, functional and non-functional requirements, agreed scale (DAU, QPS, storage)
• Step 2 (~15-20 min): high-level diagram with APIs, services, data stores, data flow arrows
• Step 3 (~15-20 min): design the 2-3 hardest or most critical components in detail
• Step 4 (~5 min): tradeoffs, bottlenecks, future improvements
• Never skip Step 1 — ambiguous scope wastes all downstream effort; get explicit agreement on assumptions

Code applications:

See references/four-step-process.md when running a design end-to-end — per-stage time allocation, example clarifying questions, and tips for each of the four steps.

2. Back-of-the-Envelope Estimation

Core concept: Use powers of two, latency numbers, and simple arithmetic to estimate QPS, storage, bandwidth, and server count before committing to an architecture.

Why it works: Estimation prevents over-provisioning (wasted money) and under-provisioning (outages under load). A 2-minute calculation can save weeks of rework.

Key insights:

• Powers of two: 2^10 ≈ 1 thousand, 2^20 ≈ 1 million, 2^30 ≈ 1 billion, 2^40 ≈ 1 trillion
• Latency: memory read ~100 ns, SSD read ~100 us, disk seek ~10 ms, same-datacenter round trip ~0.5 ms, cross-continent ~150 ms
• Availability nines: 99.9% = 8.77 hours downtime/year; 99.99% = 52.6 minutes/year
• QPS: DAU x actions-per-day / 86,400 seconds; peak is typically 2-5x average
• Storage: records-per-day x record-size x retention
• Round aggressively — the goal is order of magnitude, not precision

Code applications:

See references/estimation-numbers.md when sizing a system — full latency table, availability-nines table, and worked QPS/storage/bandwidth calculations.

3. Building Blocks

Core concept: Scalable systems are assembled from a standard toolkit: DNS, CDN, load balancers, reverse proxies, application servers, caches, message queues, and consistent hashing.

Why it works: Each block trades one cost for another (a cache trades freshness for read speed; a queue trades latency for decoupling), so introduce a block only once its specific bottleneck appears — adding all of them up front just multiplies failure modes.

Key insights:

• Load balancers: L4 (transport layer — fast, simple) vs L7 (application layer — content-aware routing)
• Cache layers: client, CDN, web server, application (Redis/Memcached), database query cache
• Cache strategies: cache-aside (app manages), read-through, write-through (synchronous), write-behind (asynchronous)
• Message queues (Kafka, RabbitMQ, SQS): decouple producers from consumers, absorb spikes, enable async processing
• Consistent hashing: distributes keys across nodes with minimal redistribution when nodes change

Code applications:

See references/building-blocks.md when choosing components — how each of DNS, CDN, load balancers, caching strategies, message queues, and consistent hashing works and when to introduce it.

4. Database Design and Scaling

Core concept: Choose SQL vs NoSQL based on data shape and access patterns; scale vertically first, then horizontally (replication and sharding) when vertical limits are reached.

Why it works: The database is usually the first bottleneck. Understanding replication, sharding, and denormalization tradeoffs delays expensive re-architectures and makes growth deliberate.

Key insights:

• Vertical scaling is simpler but has a ceiling; horizontal is harder but nearly unlimited
• Replication: leader-follower (one writer, many readers) for read-heavy; multi-leader for multi-region writes
• Sharding: hash-based (even distribution, hard range queries), range-based (easy ranges, hotspot risk), directory-based (flexible, extra lookup)
• SQL for ACID transactions, joins, defined schema; NoSQL for flexible schema, horizontal scale, very high write throughput
• Denormalization trades storage and write complexity for read speed — use when reads dominate and data changes rarely
• Celebrity/hotspot problem: one hot shard needs secondary partitioning or a cache layer

Code applications:

See references/database-scaling.md when the database is the bottleneck — replication topologies, the three sharding strategies compared, denormalization tradeoffs, and a SQL-vs-NoSQL selection guide.