Saving Millions From Millions - Navigating Towards Cost-Efficiency in Pinterest's Spark Jobs
Databricks via YouTube
Learn AI, Data Science & Business — Earn Certificates That Get You Hired
You’re only 3 weeks away from a new language
Overview
Google, IBM & Meta Certificates – 40% Off
One plan covers every Professional Certificate on Coursera.
Unlock All Certificates
Learn how Pinterest achieved tens of millions of dollars in cost savings while running millions of Apache Spark jobs monthly in this conference talk from Databricks. Discover analytical methodologies for identifying performance bottlenecks in large-scale Spark operations and explore technical solutions that address cost-efficiency challenges at massive scale. Examine strategies for extracting insights from billions of collected metrics and understand how remote shuffle services can address shuffle slowness, improve memory utilization, and reduce costs while managing hundreds of millions of pods. Gain practical knowledge about maintaining infrastructure cost efficiency to support rapid business growth and explore approaches that can help tackle common cost optimization challenges in the Apache Spark community.
Syllabus
Saving Millions From Millions: Navigating Towards Cost-Efficiency in Pinterest's Spark Jobs
Taught by
Databricks