CafeGPT - Serving LLMs Like Coffee With Kubernetes
CNCF [Cloud Native Computing Foundation] via YouTube
Start speaking a new language. It’s just 3 weeks away.
Lead AI Strategy with UCSB's Agentic AI Program — Microsoft Certified
Overview
Google, IBM & Meta Certificates – 40% Off
One plan covers every Professional Certificate on Coursera.
Unlock All Certificates
Learn the fundamentals of serving Large Language Models (LLMs) using Kubernetes through an engaging coffee shop analogy in this 26-minute conference talk. Explore how Kubernetes has become the standard platform for LLM workloads while understanding the core concepts of LLM inference, efficient deployment strategies, and GPU scheduling without getting overwhelmed by the rapidly evolving ecosystem. Discover how to decouple fundamental principles from the diverse features offered by various Kubernetes-based solutions today. Master the intersection of Kubernetes and LLM inference systems through practical insights that make complex concepts accessible, all while learning parallels to running a successful cafe operation.
Syllabus
CafeGPT: Serving LLMs Like Coffee With Kubernetes - Madhav Jivrajani & Kartik Ramesh, UIUC
Taught by
CNCF [Cloud Native Computing Foundation]