AI Inference Without Boundaries - Dynamic Routing With Multi-Cluster Inference Gateway
CNCF [Cloud Native Computing Foundation] via YouTube
PowerBI Data Analyst - Create visualizations and dashboards from scratch
Learn EDR Internals: Research & Development From The Masters
Overview
AI, Data Science & Cloud Certificates from Google, IBM & Meta — 40% Off
One plan covers every Professional Certificate on Coursera. 40% off Coursera Plus Annual.
Unlock All Certificates
Learn how to overcome GPU scarcity and scale AI inference workloads across multiple Kubernetes clusters in this 30-minute conference talk from CNCF. Discover the Multi-Cluster Inference Gateway, an open-source solution that dynamically routes AI inference traffic to available GPU resources across distributed clusters using Gateway API and multi-cluster patterns. Explore practical deployment strategies for maximizing GPU utilization, optimizing costs, and maintaining high availability for AI workloads that exceed single-cluster capacity. Gain insights into real-world implementation examples that demonstrate how to minimize latency while scaling AI serving infrastructure beyond traditional cluster boundaries, enabling intelligent traffic distribution based on resource availability across your distributed AI infrastructure.
Syllabus
AI Inference Without Boundaries: Dynamic Routing With Multi-Cluster In... Rob Scott & Daneyon Hansen
Taught by
CNCF [Cloud Native Computing Foundation]