Get 20% off all career paths from fullstack to AI
AI Engineer - Learn how to integrate AI into software applications
Overview
Google, IBM & Meta Certificates – 40% Off
One plan covers every Professional Certificate on Coursera.
Unlock All Certificates
Explore the design and implementation of Snowcannon, a synchronous data replication system for FoundationDB (FDB), in this conference talk. Learn how Snowflake addresses the limitations of FDB's current asynchronous disaster recovery solution by developing a distributed queueing system that enables synchronous transaction replication. Discover the benefits of Snowcannon, including its ability to replay rollbacks and redos, support maintenance of the secondary cluster without primary downtime, and ensure zero data loss recovery from backups. Delve into the replication protocol, failover switch, queueing architecture, and quorum protocol that form the core of Snowcannon's design. Gain insights into the challenges of building a distributed system, including the addition of new machine classes and roles. Understand how Snowcannon meets Snowflake's metadata requirements and compare it to other replication solutions.
Syllabus
Intro
OUTLINE
SNOWFLAKE REQUIREMENTS
COMPARISON OF REPLICATION SOLUTIONS
HOW TO BUILD A DISTRIBUTED SYSTEM
WE ADD A NEW MACHINE CLASS
WE ADD SUPPORT TO RECRUIT THE NEW ROLE
ADD SUPPORT TO THE WORKER
CONCLUSIONS
HIGH LEVEL VIEW OF ARCHITECTURE
SNOWFLAKE DATA REPLICATION
SNOWCANNON SWITCH
CHALLENGES WITH PRODUCER RECOVERY
SnowCannon, designed for Snowflake Metadata Requirements
Taught by
Linux Foundation