Into the Black Box: Observability in the Age of LLMs

Keynote talk exploring how to apply observability principles to Large Language Models (LLMs) in production environments. Learn how to treat LLMs as architectural black boxes similar to APIs and databases, but with unique challenges due to their probabilistic nature. Christine Yen, CEO and Cofounder of Honeycomb, discusses practical strategies for implementing observability best practices, including Service Level Objectives (SLOs), when dealing with the inherently unpredictable outputs of LLMs. Discover approaches for addressing performance challenges, accuracy issues, and user confusion in AI-enhanced products. This presentation equips developers and engineers with the confidence to navigate the complexities of building and maintaining LLM-powered features in production, even without specialized ML or AI expertise.