Evaluating Quality and Improving LLM Products at Scale

Explore strategies for evaluating and enhancing large language model (LLM) products at scale in this 15-minute conference talk by Austin Bell at the AI in Production Conference. Learn how to measure the impact of prompt changes and pre-processing techniques on LLM output quality, enabling confident deployment of product improvements. Gain insights from Bell's experience as a Staff Software Engineer at Slack, focusing on developing text-based ML and Generative AI products. Discover methods to ensure consistent enhancement of generative products through effective evaluation and measurement techniques.