Overview
Coursera Flash Sale
40% Off Coursera Plus for 3 Months!
Grab it
Learn about the architecture, design patterns, and operational challenges of Dropbox's exabyte-scale blob storage system in this technical conference talk. Explore the Object Storage Device (OSD) architecture, including buckets, volumes, and extents, while understanding how blobs are located within the storage system. Dive into erasure coding techniques and their optimizations for specific workloads, with a focus on cold storage benefits. Examine the release cycle, verification processes, and operational aspects including background traffic management, handling failures, and executing migrations. Gain insights into forecasting methodologies and practical solutions for managing storage systems at massive scale through real-world examples from Dropbox's Magic Pocket infrastructure.
Syllabus
Intro
Magic Pocket: Exabyte scale blob storage system
Some numbers
OSD (Object Storage Device)
Buckets, volumes, extents
Find a blob in object storage dev
Erasure coding
Erasure codes
Can we do better?
Observations
Optimize for this workload
Cold storage wins
Release cycle
Verifications
Operations: background traffic
Operations: failures
Operations: migrations
Forecasting
Conclusion
Taught by
InfoQ