Dynamic GPU Provisioning in Kubernetes With Composable Disaggregated Infrastructure

Explore dynamic GPU provisioning in Kubernetes environments through this 41-minute conference talk from the Linux Foundation that introduces Composable Disaggregated Infrastructure (CDI) as a solution to GPU underutilization challenges in modern data centers. Learn how the surge in AI/ML workloads has created unprecedented demand for expensive GPUs while conventional server architectures struggle with static allocation leading to inefficient resource usage and increased costs. Discover CoHDI, an innovative open-source project that leverages CDI principles to physically disaggregate resources like CPUs, memory, and GPUs, then dynamically compose them through software definition within Kubernetes clusters. Understand how CoHDI integrates with Kubernetes Dynamic Resource Allocation (DRA) features to enable on-demand attachment and detachment of GPU devices in response to Pod requests. Examine the three core components of the CoHDI architecture: the Composable DRA Driver that provides device information from hardware-disaggregated resource pools, the Dynamic Device Scaler (DDS) that makes intelligent scaling decisions based on Pod requirements, and the Composable Resource Operator that executes actual scaling operations. Gain insights into practical implementation through architectural explanations and demonstrations that showcase how this technology can revolutionize GPU resource efficiency in the AI era.