No Rose Without a Thorn - Obstacles to Successful LLM Deployments

Explore the challenges and pitfalls of deploying Large Language Models (LLMs) in production environments in this insightful 10-minute talk by Tanmay Chopra, a machine learning engineer at Neeva. Delve into the complexities that arise when moving beyond demos and MVPs to real-world applications. Examine both infrastructural and output-focused issues, including foundational model downtime, latency concerns, output variability, and prompt injections. Gain valuable insights into potential solutions for these obstacles, equipping yourself with the knowledge to navigate the intricacies of LLM deployment in both direct-to-user and intermediate layer scenarios.