Monitoring GenAI Applications

Learn how to implement effective monitoring and observability for GenAI applications through this conference talk that addresses the critical challenges of taking AI prototypes to production. Discover Project Monocle, an open source observability solution built under the Linux Foundation AI&Data that enables proactive monitoring of AI applications and their cloud infrastructure with minimal code changes. Explore how this community-driven project, built on OpenTelemetry, provides out-of-the-box support for various GenAI technology components including LLMs and vector stores, allowing developers to generate OpenTelemetry-compatible traces and spans without extensive custom code decoration. Understand the consistent format Monocle uses to describe AI entities and events like prompts and responses, and see practical demonstrations of integrating Monocle's Python and TypeScript libraries into applications across personal development environments and cloud deployments. Gain insights into solving the complex monitoring challenges that arise in the rapidly evolving GenAI landscape and learn how to bridge the gap between development prototypes and production-ready AI applications.