Saving Millions From Millions - Navigating Towards Cost-Efficiency in Pinterest's Spark Jobs
Databricks via YouTube
AI Engineer - Learn how to integrate AI into software applications
The Private Equity Associate Certification
Overview
Google, IBM & Meta Certificates — All 10,000+ Courses at 40% Off
One annual plan covers every course and certificate on Coursera. 40% off for a limited time.
Get Full Access
Learn how Pinterest achieved tens of millions of dollars in cost savings while running millions of Apache Spark jobs monthly in this conference talk from Databricks. Discover analytical methodologies for identifying performance bottlenecks in large-scale Spark operations and explore technical solutions that address cost-efficiency challenges at massive scale. Examine strategies for extracting insights from billions of collected metrics and understand how remote shuffle services can address shuffle slowness, improve memory utilization, and reduce costs while managing hundreds of millions of pods. Gain practical knowledge about maintaining infrastructure cost efficiency to support rapid business growth and explore approaches that can help tackle common cost optimization challenges in the Apache Spark community.
Syllabus
Saving Millions From Millions: Navigating Towards Cost-Efficiency in Pinterest's Spark Jobs
Taught by
Databricks