Efficient Scheduling of High Performance Batch Computing for Analytics Workloads with Volcano
CNCF [Cloud Native Computing Foundation] via YouTube
Learn AI, Data Science & Business — Earn Certificates That Get You Hired
PowerBI Data Analyst - Create visualizations and dashboards from scratch
Overview
AI, Data Science & Cloud Certificates from Google, IBM & Meta — 40% Off
One plan covers every Professional Certificate on Coursera. 40% off Coursera Plus Annual.
Unlock All Certificates
Explore how ING Wholesale Banking Advanced Analytics team implemented efficient scheduling for high-performance batch computing in analytics workloads using Volcano. Discover the journey of creating a centralized platform for internal data sources and large-scale computing, enabling over 300 internal projects and 2000 users to access advanced analytics capabilities. Learn about the implementation of a specialized cloud-native Kubernetes scheduler, Volcano, to optimize resource usage and maintain stability of core services. Gain insights into the custom extension developed for Apache Spark binaries, allowing dynamic allocation and hierarchical dominant resource fairness (HDRF) in multi-tenant environments. Understand how this solution enables users to leverage Volcano with Spark interactive mode in Jupyter notebooks and visualize scheduling metrics similar to the YARN UI.
Syllabus
Efficient Scheduling Of High Performance Batch Computing For... Krzysztof Adamski & Tinco Boekestijn
Taught by
CNCF [Cloud Native Computing Foundation]