Optimizing Istio Autoscaling - From Resource-Centric to Connection-Aware

Learn how to optimize Istio autoscaling by transitioning from traditional resource-centric metrics to connection-aware scaling strategies in this 28-minute conference talk. Discover why conventional CPU and request throughput metrics fail for applications handling long-lived connections such as GraphQL, WebSockets, and streaming APIs, which can appear idle to Horizontal Pod Autoscaler (HPA) even when operating near connection saturation. Explore practical insights from operating a multi-cluster Istio mesh and understand how to model traffic using active connection counts to make HPA function effectively for ingress gateway scaling. Gain hands-on knowledge about tuning node sizing and pod resource limits to eliminate cold starts and avoid bin-packing issues. Master techniques for preventing 503 errors in cross-cluster scenarios while scaling intelligently and controlling costs, all while respecting the unique characteristics of long-lived connections in production Istio environments.