Designing for Failure - Building Resilient Services and Recovery Systems

Learn essential strategies for building resilient services in this 51-minute conference talk from Conf42 DevSecOps 2024. Explore comprehensive approaches to failure-aware design, starting with fundamental concepts of redundancy across infrastructure, data, and microservices. Master both automatic and manual failover mechanisms while understanding their implementation challenges. Discover techniques for graceful degradation and shutdown, supported by real-world examples and testing scenarios. Dive into chaos engineering principles, including its benefits, challenges, and practical techniques. Gain expertise in implementing circuit breakers and automated recovery mechanisms to ensure system reliability. By the end, acquire key strategies for developing robust, always-on services that can withstand various failure scenarios.

Syllabus

00:00 Introduction to Failure-Aware Design
02:02 The Importance of Redundancy
03:12 Infrastructure Redundancy Explained
04:26 Data Redundancy Strategies
05:39 Service Redundancy in Microservices
06:44 Balancing Redundancy Costs
08:48 Failover Mechanisms
09:26 Automatic vs. Manual Failover
13:08 Challenges in Implementing Failover
15:49 Graceful Degradation
22:18 Designing for Graceful Shutdown
26:51 Challenges of Graceful Shutdown
27:27 Testing Shutdown Scenarios
28:36 Real World Examples of Graceful Shutdown
29:47 Introduction to Chaos Engineering
31:29 Techniques in Chaos Engineering
33:23 Benefits of Chaos Engineering
34:07 Challenges in Chaos Engineering
35:53 Understanding Circuit Breakers
42:20 Automated Recovery Mechanisms
49:29 Key Strategies for Building Resilient Systems