Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

CNCF [Cloud Native Computing Foundation]

Running Remote Shuffle Service to Solve Apache Spark's Dynamic Resource Allocation Challenge on Kubernetes

CNCF [Cloud Native Computing Foundation] via YouTube

Start learning Write review
Coursera Ad

Learn AI, Data Science & Business — Earn Certificates That Get You Hired

Learn More → DataCamp Ad

AI Engineer - Learn how to integrate AI into software applications

Learn More →

Overview

Google, IBM & Meta Certificates – 40% Off
One Coursera Plus subscription covers most Professional Certificates on Coursera.
Unlock All Certificates
This talk examines Spark's dynamic resource allocation challenge on Kubernetes and a remote shuffle service that moves shuffle data outside executor pods. It explains how this approach decouples storage from compute to support dynamic scaling and reduce resource contention.

Syllabus

Running Remote Shuffle Service to Solve a Well-Known Challenge for Apa... Melody Yang & Keyong Zhou

Taught by

CNCF [Cloud Native Computing Foundation]

Reviews

Start your review of Running Remote Shuffle Service to Solve Apache Spark's Dynamic Resource Allocation Challenge on Kubernetes

Start learning

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Sign up for free
Someone learning on their laptop while sitting on the floor.