Why Are Distributed Systems So Hard?

This talk explains why distributed systems are difficult, covering distributed computing history, scaling, the CAP theorem, network partitions, consistency, availability, and consensus algorithms. It also considers observability and human factors that affect system adaptability and operational reliability.

Syllabus

Introduction
Agenda
Storytime
Data Evolution
Scaling
Cloud Computing
Why Scale Horizontally
What Does It Mean To Run A Distributed System
A Node On Distributed Computing
Summary
Shared Nothing Architecture
Unreliable Message Delivery
Why Are We Fenced Off
Building Observability
What We Can Know
The Cap Theorem
C
Replication Lag
Consistency is a Spectrum
Availability is Not Binary
Partition Tolerance
Hardware
Hardware Failure
Cables
Sharks
Kevlar
Network Partitions
Resource Isolation
Process Suspension
Network Glitch
People do bad things
Why does this matter
Practical reality
The correctness result
Mitigation strategies
Consensus Algorithms
The Woods Theorem
Building Mental Models
Incident Analysis
Blameless Discussions
Mental Models
Human Failure
Alert Fatigue
User Mindsets
Designing Systems for Humans
HugOps