System design map
A typical architecture, one layer at a time. Each box opens what I've written about that component. Greyed boxes are gaps I haven't written up yet.
Common problems
Pick a problem to light up the building blocks that solve it. From Common System Design Patterns.Clients & edge
- Client
- CDN1
- Blob storage1
- Real-time updates2
Entry point
Services
Async processing
Data
Cross-cutting
Client
Browser, mobile app or another service.
CDN
CDN - Content Delivery Network Growing
Spread out servers that store copies of content. Page load will be faster as there will be a server closer to the user's location, usually. These usually store and cache content like…
Blob storage
Pre-signed urls Growing
Pre signed urls rather than streaming through api servers with bigger timeouts. This frees servers to do other things, this is supposedly a secure way to get clients direct access to blob…
Real-time updates
WebSockets vs SSE vs API polling Growing
WebSockets is 2 way open connection. SSE (Server Sent Events) is one way. API usually have a timeout of 30-60 seconds, configured by an outside system, often. If the operation you're doing…
Related notes
- WebSocket Seedling
Trade-offs
Load balancer
Load Balancer Established
Single point of contact for user requests. It does health checks for all servers to know if some have issues. Distribution: Round Robin: simple and sequential. IP hashing: allows the same user…
Related notes
- Reverse proxy vs API gateway vs load balancer Established
- Stateless vs Sticky sessions Growing
Trade-offs
API gateway
API Gateway Growing
Api gateways can combine results from different apis into one. Routing: https://www.hellointerview.com/learn/system-design/deep-dives/api-gateway.
Related notes
- Reverse proxy Seedling
- Forward proxy vs Reverse proxy vs API Gateway Established
Trade-offs
Rate limiting
API Rate limiting Growing
Django rate limiter? In memory per server. You don't get a global picture tho. Or separate service but extra latency. Most common is to have it with the api gateway. Rate limiting…
Authentication
API Authentication Growing
Maybe not straight user ID in api. Jwt json web token contains signed json data about the user, like what role they have. Every request contains a signature that the server can…
Related notes
- Managing sessions Growing
App servers
Horizontal scaling Seedling
Horizontal scaling only works if servers are stateless aka, server needs to treat the request as if the user came in here for the very first time.
Related notes
API design
API Design Seedling
Related notes
- REST vs GraphQL Growing
- GraphQL Growing
- RPC methods Growing
- REST framework Growing
Trade-offs
Resilience
API retries Growing
Good to handle the odd error. Beware of overloading already failing services with even more load. API Circuit breakers will help.
Related notes
- API Circuit breakers Growing
- API short vs long timeouts Growing
- Idempotency key Growing
Trade-offs
Configuration
Distributed configuration management Growing
Distributed configuration management, zookeeper is a bit outdated but good for this. Example: The API Gateway keeps an open connection to config server and pushes changes straight away. Other not as good…
Message queue
Message queues Established
operations too long for a live server, get feedback to users quickly. reduce load on a live server. when you want the ability to scale differently for different tasks (ie retrievers need…
When to use system queues
- operations too long for a live server, get feedback to users quickly
- reduce load on a live server
- when you want the ability to scale differently for different tasks (ie retrievers need less memory than collectors in dogs)
- reliability: keep messages to be processed in queue until workers are healthy again
Related notes
- Kafka vs SQS Growing
Trade-offs
Event-driven
Event Driven System Growing
Dealing with important events that needs to be stored forever (ie refund events) When you need replayability of events (reprocess historical data) Real-time data processing.
When to use them
- Dealing with important events that needs to be stored forever (ie refund events)
- When you need replayability of events (reprocess historical data)
- Real-time data processing
Batch / stream
Batch processing vs stream processing Growing
In old existing product data, it was stream processing, processing one spider item at a time. whereas in dogs, it was batch processing, processing 'big' sqlite files.
Related notes
- Cron jobs to fix data vs TTL Growing
Trade-offs
Distributed transactions
Distributed transactions Growing
2 phase commit and saga patterns are the most common ways to solve this. You can have a coordinator plan across services and commit all at once, but this is fragile if…
Related notes
Trade-offs
Cache
Caching Established
Cache = trade off speed vs freshness. You can also use the CDN as a cache for api results. Short term. Ok if only a few. Consider how often it changes, how…
When to use caching
Related notes
- Caching patterns Established
- Redis Growing
- Redis vs Memcached Growing
Trade-offs
Database
Database Seedling
Related notes
- How to choose a database Established
- Access pattern driven modelling Growing
- Relational Database Growing
- PostgreSQL Growing
- DynamoDB Seedling
- Cassandra Database Growing
- AWS Aurora Established
- Database types Seedling
Trade-offs
Scaling the DB
Database Sharding Established
Write-heavy: Too much data for 1 instance (storage or write per second) Read-heavy: when read replicas or caching are not enough. Global queries will be slower. Complexity to operate and keep data…
When to use sharding
- Write-heavy: Too much data for 1 instance (storage or write per second)
- Read-heavy: when read replicas or caching are not enough
Trade offs
- Global queries will be slower
- Complexity to operate and keep data consistent
Related notes
- Database Replication Seedling
- Database index Growing
- Connection Pooling Growing
- Sync between databases Growing
Search
Search systems Growing
Postgres is fine to begin with for basic search. Select where like term. When would somethint else be more relevant ? Wildcard search are slow tho, it'll scan the whole db. But…
Related notes
- ElasticSearch Growing
- Search optimised database Growing
Latency
Latency Seedling
Latency is the time to perform some action or to produce some result.
Throughput
Throughput Growing
Throughput is the number of such actions or results per unit of time. Generally, you should aim for maximal throughput with acceptable latency.
Availability
Availability Growing
Availability patterns. There are two main patterns to support high availability: fail-over and replication. Fail-over. Active-passive. With active-passive fail-over, heartbeats are sent between the active and the passive server on standby. If…
Trade-offs
Consistency
Consistency Growing
Weak consistency. After a write, reads may or may not see it. A best effort approach is taken. This approach is seen in systems such as memcached. Weak consistency works well in…
Related notes
- Consistency vs Availability Growing
- Database ACID Growing
Trade-offs
Scalability
Scalability Growing
A service is scalable if it results in increased performance in a manner proportional to resources added. Generally, increasing performance means serving more units of work, but it can also be to…
Related notes
- Horizontal scaling Seedling
Real time updates
Reach for
Notes
- WebSocket Seedling
- WebSockets vs SSE vs API polling Growing
Long running tasks
Reach for
Notes
- Message queues Established
Data contention
Reach for
Notes
- Database ACID Growing
Scaling reads
Reach for
Notes
- Caching Established
- Database Replication Seedling
Scaling writes
Reach for
Notes
- Database Sharding Established
Handling large blobs
Reach for
Notes
- Pre-signed urls Growing
Multi step process
Reach for
Notes
Proximity based services
Something something geolocalisation Postgres has extensions for this
Not written up yet.
Numbers to know
Rough orders of magnitude for back-of-the-envelope estimates. Each one links to the note it comes from.
Speed
- Memory vs disk reads
- ≈10,000× fasterWhy a cache in front of the database pays off.Caching
Throughput
- Redis throughput
- ≈100k req/sPer instance, rough order of magnitude.Redis
- PostgreSQL reads
- ≈20k queries/sSingle instance.PostgreSQL
- PostgreSQL writes
- ≈5–10k writes/sSingle primary; beyond this consider sharding or a write-optimised store.How to choose a database
- When DynamoDB starts to make sense
- > 10k writes/sHeavy writes with key-value access patterns.PostgreSQL vs DynamoDB
Storage
- Single PostgreSQL gets uncomfortable
- ≈100 GB – 1 TBMaintenance, vacuum and backups get painful; technical limit is far higher.PostgreSQL
- Aurora storage
- up to 128 TBGrows automatically in 10 GB increments.AWS Aurora
- One user row
- ≈1 KBIncluding overhead; raw fields are ~150 bytes (ids 8–16 B, email 30–60 B, bcrypt hash 60 B).How much data is a user in a database roughly