![CNCF [Cloud Native Computing Foundation]](https://www.classcentral.com/cdn-cgi/image/height=60,format=auto,quality=85/images/logos/institutions/cncf-cloud-native-computing-foundation-hz.png){kind=small}
AI, Data Science & Cloud Certificates from Google, IBM & Meta — 40% Off
One plan covers every Professional Certificate on Coursera. 40% off Coursera Plus Annual.
Unlock All Certificates
Explore the challenges and solutions of building a cloud-agnostic, privately hosted Large Language Model (LLM) serving platform on Kubernetes in this 26-minute conference talk. Discover how Predibase tackled the complexities of hosting LLMs, including their large size and GPU resource requirements. Learn about the architecture of their control plane and dataplane, secured with an Istio service mesh, and the implementation of KEDA for event-driven auto-scaling to support serverless inference of open-source models. Gain valuable insights into deploying LLMs and acquire practical knowledge on applying tools and techniques for your own organization's LLM hosting needs.
![CNCF [Cloud Native Computing Foundation]](https://www.classcentral.com/cdn-cgi/image/height=40,format=auto,quality=85/images/logos/institutions/cncf-cloud-native-computing-foundation-hz.png){kind=small}
