Taming Data Skew in Production ML Pipelines

Learn to identify, understand, and solve data skew challenges in production machine learning pipelines through this 24-minute conference talk from Conf42 ML 2026. Discover how data skew manifests in Apache Spark environments through symptoms like stragglers, shuffle spills, and idle executors, using real-world examples from Roku's large-scale operations. Explore the fundamental nature of data skew and why it breaks parallelism, examining specific scenarios involving power users, hot keys, and power-law distributed data. Understand where skew impacts ML pipelines across recommendation systems, classification tasks, and computer vision applications. Investigate three primary root causes: natural imbalances from real-world events, join-key and aggregation skew during feature engineering, and computational skew from NLP processing, embeddings, and heavy transformations. Master three key mitigation strategies including repartitioning techniques with their limitations, key salting methods to split hot keys for significant runtime improvements, and broadcast joins to eliminate massive shuffles. Gain insights into selecting appropriate solutions for different scenarios and explore emerging AI approaches for predicting skew before it impacts production systems.