In this topic

• How much data is a user in a database roughly🌿Growing

  1KB ish. Some common fields: But what takes most space is database overhead, index, page, etc.

• Immutable data vs mutable data🌿Growing

  An image or a song would be immutable data. The access pattern for those is a lot of reads, very few writes. These would be best stored in a system where it's…

• ElasticSearch🌿Growing

  It builds an inverted index to make searching documents by terms quickly. Example: taylor: (event1, event2) Tokenization in Elasticsearch is the process of breaking a stream of text characters down into smaller,…

• Database ACID🌿Growing
• Database Replication🌱Seedling

  Synchronous is slower, but ensures stronger consistency.

• Database Sharding🌳Established

  Write-heavy: Too much data for 1 instance (storage or write per second) Read-heavy: when read replicas or caching are not enough. Global queries will be slower. Complexity to operate and keep data…

• Database index🌿Growing

  Removes the need for a slow sequential scan. B-tree is the most popular, allows range queries with almost O(1). Hash index (key mapping) is definitely fast and O(1) but can't do range…

• Relational Database🌿Growing
• How to choose a database🌳Established

  PostgreSQL. Strengths: transactions, joins, constraints, flexible queries. Weaknesses: horizontal scale/sharding complexity, not ideal for huge analytical scans. DynamoDB. Strengths: massive scale, predictable low-latency key access, operational simplicity. Weaknesses: access-pattern-driven modelling, limited relational…

• Distributed transactions🌿Growing

  2 phase commit and saga patterns are the most common ways to solve this. You can have a coordinator plan across services and commit all at once, but this is fragile if…

• Access pattern driven modelling🌿Growing

  Rather than modelling your data model based on what it is (relations, etc.). Model based on how the application will need to query it (key value, top 50, complex search, etc.) Optimise…

• Redis🌿Growing

  Redis is great for data that needs to be accessed by key quickly. Especially if that data is temporary with the TTL feature, like a cache a ticket locking system. It would…

• PostgreSQL🌿Growing

  read: 20k queries per second. write: 10k writes per second.

• DynamoDB🌱Seedling
• AWS Aurora🌳Established

  Separates compute from storage. Leverages s3 as elastic storage, making it scale way beyond what traditional dbs can do. For example, for PostgreSQL it starts struggling around 100GB to 1TB. With aurora,…

• Cassandra Database🌿Growing

  Cassandra is masterless. Cassandra has no Database ACID support. Bias towards Availability. Write heavy. Zero downtime needed (in traditional master/slave, there is a small downtime if master is down for the failover…

• Redis vs Memcached🌿Growing

  Memcached is supposedly a bit faster as it is multi threaded. Redis has better support of more data types. Redis has optional disk persistence.

• PostgreSQL vs MySQL🌿Growing

  MySQL is thread per connection, where PostgreSQL is process per connection. MySQL is seemingly faster for lots of concurrent simple reads. PostgreSQL takes up more memory, but handles complex queries better with…

• PostgreSQL vs DynamoDB🌿Growing

  When I'd use DDB: if key value access is enough. I need heavy write (more than 10k writes per second) automatic and managed sharding. What you're giving up with postgres: relations and…

• Database types🌱Seedling