Tracing the Untraceable - Full-Stack Observability for LLMs and Agents

Learn how to implement comprehensive observability for Large Language Model applications and AI agents without requiring manual instrumentation in this 14-minute conference talk. Discover the critical monitoring challenges that emerge when organizations rush to productionize LLM workflows, including performance volatility, quality drift, and security vulnerabilities that traditional observability tools cannot detect. Explore practical techniques for breaking out of the "LLM black box" to gain visibility into token usage patterns, response latency metrics, data exposure risks in prompts, and model execution failures across both commercial LLM platforms and AWS Bedrock environments. Master zero-instrumentation monitoring approaches that provide instant troubleshooting capabilities for complex AI-powered applications, enabling you to maintain reliability and security as you scale LLM-based systems in production environments.