Swimming Through Molasses - How Scale Makes Distributed Systems Slower

Explore the counterintuitive phenomenon where distributed systems become slower as they scale up in this 38-minute conference talk from GOTO Chicago 2024. Discover why adding more machines to a system doesn't always improve performance and sometimes creates a "swimming through molasses" effect. Learn about Scale-Originated Coordination Overhead (SOCO) and how coordination and synchronization protocols introduce costly overhead that can negate the benefits of horizontal scaling. Examine experimental evidence from both academic research and industrial case studies that demonstrate these performance paradoxes. Understand the CALM theorem and when distributed consistency becomes manageable, explore Dedalus as a temporal logic programming language for distributed systems, and see how datalog concepts translate to dataflow architectures. Gain practical mental models and conceptual frameworks for structuring distributed system problems to prevent runtime slowdowns, with specific insights into coordination costs and synchronization challenges. Access comprehensive resources including academic papers, recommended books on distributed systems patterns, and links to related technologies like CockroachDB, Kubernetes, Apache Kafka, and various distributed databases.