Optimized Scheduling for Big Data Workloads - The Why, What and How of K8s Schedulers

Explore the complexities of scheduling stateful workloads in Kubernetes through this 35-minute conference talk from CNCF. Discover why the default Kubernetes scheduler often falls short when handling databases, caching systems, and message brokers that require persistent storage, stable network identities, and precise resource allocations. Learn about the fundamental challenges these big data workloads present compared to stateless applications, including potential performance degradation, resource inefficiencies, and operational instability. Examine the three essential components of effective scheduling through a comprehensive breakdown of the why, what, and how of popular Kubernetes schedulers. Gain practical insights into adapting Kubernetes scheduling mechanisms to meet the specialized demands of stateful applications and big data workloads, equipping yourself with strategies to optimize performance and maintain operational stability in cloud-native environments.