Database Internals A deep-dive into how distributed data systems work

For software developers, reliability engineers, and architects who rely on databases but find their inner workings a mystery, "Database Internals" demystifies the complex world of storage engines and distributed data systems. Author Alex Petrov provides a concise yet thorough tour of fundamental concepts, from disk-based B-Trees and log-structured merge-trees to distributed algorithms for failure detection, leader election, replication, and consensus. By breaking down the trade-offs in storage design (buffering, immutability, ordering) and the challenges of distributed computing (CAP theorem, consistency models), this book equips you with the mental models needed to choose the right database, troubleshoot performance issues, and build more resilient applications.

What it argues

This model outlines how key architectural choices in database system design, particularly concerning storage engines and distribution mechanisms, lead to specific intermediate system behaviors (like amplification and overhead) and ultimately determine overall system performance outcomes like workload performance, availability, and consistency.

Key ideas it contributes

• Storage Structure Choice — The fundamental architectural decision regarding the data structure used by the storage engine to organize, store, and access data on a single node, primarily the choice between in-place update structures (like B-Trees) and append-only log-structured organizations (like LSM-Trees).
• Replication & Consistency Model — The set of rules and algorithms governing how data is copied across multiple nodes and the guarantees provided to clients about the visibility and ordering of updates, ranging from strong models (e.g., consensus-based) to weaker ones (e.g., eventual consistency with quorums).
• Write Amplification — The total amount of data physically written to the storage device relative to the amount of data the application logically requested to write. High write amplification can degrade performance and reduce the endurance of flash storage.
• Read Amplification — The number of physical read operations or blocks that must be accessed from storage to satisfy a single logical read request. High read amplification increases read latency.
• Coordination Overhead — The resources (time, network bandwidth, CPU) expended on communication and synchronization between distributed nodes to maintain a consistent state or reach agreement.
• Workload Performance — The efficiency of the database system in executing read and write operations under a specific workload, typically measured in terms of throughput and latency.
• System Availability — The degree to which the database system is operational and responsive to client requests, especially in the presence of node or network failures.
• Data Consistency Guarantees — The contractual promise made by the system to the application about the ordering and visibility of read and write operations across replicas.