Building a Fine-Grained and Intelligent Resource Management System on Kubernetes

Explore a comprehensive conference talk on building an advanced resource management system for Kubernetes. Discover how Katalyst, developed by ByteDance, addresses limitations in vanilla Kubernetes resource management. Learn about innovative techniques for improving resource utilization, including online-offline workload colocation, GPU-sharing scheduling with fine-grained allocation, and topology-aware scheduling optimized for AI workloads. Gain insights into practical methods for enhancing resource efficiency, such as node over-commitment, specification recommendation, and tidal colocation. Understand how these improvements can significantly boost performance in scenarios like AI inference, distributed model training, and large-scale online services while maintaining service level objectives.