On-Call the Easy Way With Agents - Debugging API Traffic Systems Over Service Mesh

Learn how to streamline on-call debugging and incident response for API traffic systems over service mesh architectures like Istio using AI agents in this 24-minute conference talk. Discover how Intuit developed the Mesh Debug Deputy, an AI agent that integrates with observability tools like Splunk and Wavefront, leverages RAG (Retrieval-Augmented Generation) over internal documentation, and executes runbook commands including kubectl and istioctl to automate complex debugging tasks. Explore the architecture and implementation of an intelligent system that can analyze PagerDuty alerts, gather data from multiple sources, perform diagnostic steps, and execute remediation commands to resolve issues automatically. Understand how this approach significantly reduces mean time to resolution (MTTR) and creates a more efficient on-call experience by automating routine debugging workflows that traditionally require manual intervention from engineers. Gain insights into building AI-powered operations tools that can interpret alert descriptions, correlate data across multiple observability platforms, and take actionable steps to resolve service mesh-related incidents.