Completed
Envoy, take the wheel: Realtime adaptive circuit breaking Protecting Microservices with Adoptive Concurrency
Class Central Classrooms beta
YouTube videos curated by Class Central.
Classroom Contents
Envoy, Take the Wheel - Real-time Adaptive Circuit Breaking
Automatically move to the next video in the Classroom when playback concludes
- 1 Envoy, take the wheel: Realtime adaptive circuit breaking Protecting Microservices with Adoptive Concurrency
- 2 Agenda • Simulating traffic. • Concurrency and circuit breaking. • Adaptive concurrency control. • Tales from lyn infra
- 3 Simulations.
- 4 Full Simulation Output.
- 5 Simulating Traffic Overload.
- 6 Concurrency. There's some number of requests a system can service at any given time
- 7 Really, bursts are fine.
- 8 Envoy concurrency limits via circuit breaking. Cluster Max Connections Limit number of retries outstanding
- 9 Revisiting Traffic Overload!
- 10 Properly configuring circuit breakers. Understand service limitations
- 11 Properly configuring circuit breakers is hard. Service owners shouldn't have to profile concurrency
- 12 Configuring limits in an ideal world... No manual configuration
- 13 Envoy Filters
- 14 HTTP Connection Manager Network Filter Parses raw bytes over connection • Bytes are converted into HTTP messages • Allows Envoy to operate at a higher level of abstraction Contains an HTTP filter cha…
- 15 The gradient
- 16 Concurrency Limit Behavior.
- 17 There's more...
- 18 Adaptive Concurrency Control Filter
- 19 Adaptive Concurrency in Action
- 20 Adaptive Concurrency i ction
- 21 Steady-state with min_concurrency too low.
- 22 Configuring Adaptive Concurrency jitter
- 23 Experiences @lyR: Our default settings Our default adaptive concurrency settings
- 24 Experiences @ lyR: General observations Almost all adaptive concurrency events are due to service degradation, not spikes in RPS. Bad deploys Upstream dependency outages/delays.
- 25 Experiences @ lyR: The bad deploy Changes went out that resulted in elevated CPU utilization
