AI Agent Observability - Monitoring and Debugging Complex Autonomous Systems

This conference talk from MLOps World explores the critical field of AI Agent Observability, presented by Claire Longo, Lead AI Researcher at Comet. Discover how to monitor, debug, and ensure reliability in increasingly autonomous and complex AI systems. Learn best practices for tracking agent behavior, understanding decision-making processes, and identifying failure points or biases. The presentation covers essential techniques including logging traces, monitoring strategies, and LLM evaluation metrics that provide visibility into agent operations, along with novel approaches inspired by Reinforcement Learning. Through real-world case studies, examine how AI Observability enhances AI-driven systems in today's industry and gain practical strategies to implement Comet Opik for observability in your own AI agents. Benefit from Longo's decade of experience across data science, machine learning, and GenAI as she shares insights from her work at organizations including DOE National Laboratories, Twilio, Opendoor, and Arize AI.