LLM Serving - The 4 Hard Truths No One Tells You

Discover the critical engineering challenges of scaling Large Language Models to production in this 50-minute conference talk by Charlotte Qi from Meta's AI Infrastructure team. Learn about the four hard truths that engineers face when building real-world LLM serving infrastructure, going beyond simple model deployment to address optimization for speed, reliability, and cost at unprecedented scale. Explore the engineering minefield of production LLM systems and gain insights from Meta's experience in handling AI infrastructure challenges. Understand the complexities involved in serving LLMs effectively in enterprise environments and the practical considerations that are often overlooked in theoretical discussions about AI deployment.