Scale Your Batch, Big Data, and AI Workloads Beyond the Kubernetes Scheduler

Explore the challenges and solutions for scaling batch, big data, and AI workloads on Kubernetes in this 25-minute DevConf.US 2024 conference talk. Delve into the limitations of the traditional Kubernetes scheduler when handling resource-intensive, highly-coupled processes. Discover recent innovations in the Kubernetes ecosystem designed to address issues such as resource fragmentation, quota management, auto-scaling, and priority handling. Compare projects like Karmada, Koordinator, Kueue, MCAD, Volcano, and YuniKorn, examining their design choices and trade-offs. Gain insights to help you select the most suitable solution for optimizing cluster utilization and managing batch workloads effectively in your Kubernetes environment.