Learn Excel & Financial Modeling the Way Finance Teams Actually Use Them
AI Engineer - Learn how to integrate AI into software applications
Overview
Google, IBM & Meta Certificates — All 10,000+ Courses at 40% Off
One annual plan covers every course and certificate on Coursera. 40% off for a limited time.
Get Full Access
Learn about the architecture, design patterns, and operational challenges of Dropbox's exabyte-scale blob storage system in this technical conference talk. Explore the Object Storage Device (OSD) architecture, including buckets, volumes, and extents, while understanding how blobs are located within the storage system. Dive into erasure coding techniques and their optimizations for specific workloads, with a focus on cold storage benefits. Examine the release cycle, verification processes, and operational aspects including background traffic management, handling failures, and executing migrations. Gain insights into forecasting methodologies and practical solutions for managing storage systems at massive scale through real-world examples from Dropbox's Magic Pocket infrastructure.
Syllabus
Intro
Magic Pocket: Exabyte scale blob storage system
Some numbers
OSD (Object Storage Device)
Buckets, volumes, extents
Find a blob in object storage dev
Erasure coding
Erasure codes
Can we do better?
Observations
Optimize for this workload
Cold storage wins
Release cycle
Verifications
Operations: background traffic
Operations: failures
Operations: migrations
Forecasting
Conclusion
Taught by
InfoQ