A Deep Dive on Supporting Multi-Instance GPUs in Containers and Kubernetes

Explore a comprehensive conference talk on supporting Multi-Instance GPUs (MIG) in containers and Kubernetes. Dive deep into the technical aspects of implementing MIG support, a feature of NVIDIA Ampere GPUs that allows partitioning a GPU into smaller "MIG Devices". Learn about the integration of MIG with containers, the challenges faced in building Kubernetes support, and how to utilize this technology. Discover the open-source solutions developed by NVIDIA, including the container toolkit stack and k8s-device-plugin. Gain insights into best practices for distributing MIG devices across a Kubernetes cluster and managing their lifecycle on nodes. The presentation covers topics such as GPU scaling in Kubernetes, the NVIDIA Container Toolkit, GPU allocation to pods, system-level interfaces for MIG, and challenges in MIG partitioning. Conclude with a summary of key points and future implications for GPU virtualization in cloud-native environments.