Pinterest's Approach to Real-Time ML Experimentation Using Ray

Learn how Pinterest transformed their machine learning experimentation workflow from a critical bottleneck into a fast, iterative process using Ray's real-time streaming capabilities in this 30-minute conference talk from Ray Summit 2025. Discover how Pinterest's engineering team tackled the fundamental challenge of dataset curation and label generation that was severely limiting developer velocity and forcing costly trade-offs between expensive compute-intensive backfills and weeks-long waits for experimental data. Explore two core use cases that exemplified these problems: sampling strategy exploration where training data composition directly impacts model performance, and downstream engagement label generation requiring complex multi-parameter definitions that created vast, previously infeasible hyperparameter search spaces. Understand the architectural shift from static dataset generation to a real-time streaming paradigm built on Ray, where sampling and label-generation logic moved directly into the training dataloader to enable on-the-fly data processing. Examine the transformative results achieved through this Ray-powered pipeline, including 10× faster sampling updates for ranking models, engagement label iteration reduced from 6 weeks to 3 days, widespread adoption across multiple teams, cost-effective joining of massive embedding features and multi-day datasets through Ray's bucket join capabilities, and overall savings of hundreds of thousands of dollars in compute costs. Gain insights into how Pinterest unified sampling, labeling, and training into a single scalable pipeline that converted dataset iteration from a fundamental constraint into a catalyst for rapid model improvement.