Serving the Future: KServe's Next Chapter Hosting LLMs and GenAI Models

Join Alexa Griffith and Tessa Pham from Bloomberg in this 29-minute conference talk exploring KServe's evolution for hosting Large Language Models (LLMs) and Generative AI models. Discover how KServe has transformed from a simple ML model deployment platform on Kubernetes to a comprehensive solution for managing advanced AI workloads at scale. Learn about KServe's latest features specifically designed for generative AI, including enhanced serving runtimes, scalability improvements, and integration strategies across hybrid environments. Gain practical insights from the speakers' dual perspective as both KServe maintainers and practitioners who implement these solutions in Bloomberg's production clusters. The presentation includes fun drawings to illustrate concepts and shares valuable real-world experiences and lessons learned when deploying and scaling generative models in production.