Kueue: Kubernetes-Native Job Queueing for Batch Workloads

This talk explains how Kueue brings queueing to Kubernetes batch workloads. It covers queue and workload APIs, resource sharing across namespaces and cluster queues, and integration with existing controllers and autoscaling.

Syllabus

Introduction
Problem description
What is a job
Batch workloads
Flexible on multiple dimensions
The problem
Microbatch jobs
What is Job Queueing
What we want to solve
Why propose a new controller
The concept of a queue
Design principle
Job API
Resource Model
Multiple namespaces
Multiple cluster queues
Borrowing cohorts
Queue API
Cluster Queue API
Dakota
Borrowing Cohort
How it works
suspend
queuing controller
existing controller
handling custom workloads
workload API
workload controller
autoscaling
why
integration
capacity requests
pod management
release
questions
how does the queue interact with index jobs