![CNCF [Cloud Native Computing Foundation]](https://www.classcentral.com/cdn-cgi/image/height=60,format=auto,quality=85/images/logos/institutions/cncf-cloud-native-computing-foundation-hz.png){kind=small}
AI, Data Science & Cloud Certificates from Google, IBM & Meta — 40% Off
One plan covers every Professional Certificate on Coursera. 40% off Coursera Plus Annual.
Unlock All Certificates
Keynote presentation exploring how traditional load balancing methods fail when applied to Large Language Model (LLM) serving in Kubernetes environments. Discover new Kubernetes APIs specifically designed for LLM workload routing that accommodate the unique computational challenges posed by varying prompt lengths, model differences, and autoregressive processing. Learn how these APIs integrate with Gateway API and can be implemented across various Gateway API implementations for streamlined LLM routing support. The speakers demonstrate the practical application of this project through real-world examples, showcasing significant efficiency improvements. This 16-minute talk from KubeCon + CloudNativeCon Europe features Clayton Coleman, Distinguished Engineer at Google, and Jiaxin Shan, Software Engineer at Bytedance, as they introduce a new era of cloud-native LLM infrastructure management.
![CNCF [Cloud Native Computing Foundation]](https://www.classcentral.com/cdn-cgi/image/height=40,format=auto,quality=85/images/logos/institutions/cncf-cloud-native-computing-foundation-hz.png){kind=small}
