AI Development Lifecycle: Key Changes and Challenges with LLMs

Explore a 28-minute conference talk examining the key differences between building LLM-based models versus traditional machine learning approaches. Delve into the contrasting aspects of development, where Proof of Concepts have become significantly simpler while evaluation poses greater challenges. Learn from real-world experiences in implementing LLM products, with special focus on a RAG (Retrieval Augmented Generation) tool developed for a medical company. Master essential yet often overlooked development workflow components including dataset collection, evaluation methods, and monitoring strategies. Discover new tools and techniques for both manual and automatic evaluation using LLM as a judge, and understand how product experts can effectively collaborate with technical teams to build robust LLM-based solutions.