Productionizing LLMs at Scale for Real-World Commerce

This technical case study presentation from Ray on the Road NYC 2025 explores how Mirakl revolutionized e-commerce catalog onboarding by transforming it from a weeks-long process into an automated, AI-powered system using fine-tuned large language models. Discover the architecture and operational decisions behind Mirakl's production LLM platform called Catalog Transformer, which significantly accelerates how marketplace sellers get products online. Learn about their implementation of LoRA adapters for efficient updates, elastic autoscaling for handling traffic fluctuations, and dynamic model serving on Anyscale using LLaMa 3.1 8B. The 25-minute talk provides valuable insights for ML engineers, platform architects, and product teams working with production LLM applications, covering practical architecture choices, lessons from scaling language models in production, and cost optimization strategies that enable sustainable AI-powered business processes in e-commerce.