![CNCF [Cloud Native Computing Foundation]](https://www.classcentral.com/cdn-cgi/image/height=60,format=auto,quality=85/images/logos/institutions/cncf-cloud-native-computing-foundation-hz.png){kind=small}
Google, IBM & Meta Certificates — All 10,000+ Courses at 40% Off
One annual plan covers every course and certificate on Coursera. 40% off for a limited time.
Get Full Access
Keynote presentation exploring how traditional load balancing methods fail when applied to Large Language Model (LLM) serving in Kubernetes environments. Discover new Kubernetes APIs specifically designed for LLM workload routing that accommodate the unique computational challenges posed by varying prompt lengths, model differences, and autoregressive processing. Learn how these APIs integrate with Gateway API and can be implemented across various Gateway API implementations for streamlined LLM routing support. The speakers demonstrate the practical application of this project through real-world examples, showcasing significant efficiency improvements. This 16-minute talk from KubeCon + CloudNativeCon Europe features Clayton Coleman, Distinguished Engineer at Google, and Jiaxin Shan, Software Engineer at Bytedance, as they introduce a new era of cloud-native LLM infrastructure management.
![CNCF [Cloud Native Computing Foundation]](https://www.classcentral.com/cdn-cgi/image/height=40,format=auto,quality=85/images/logos/institutions/cncf-cloud-native-computing-foundation-hz.png){kind=small}
