Managing Human in the Loop Systems Without Burning Out Your Engineers

Explore strategies for effectively managing human-in-the-loop systems in machine learning without overwhelming engineering teams in this conference talk from MLOps World. Discover how Pinterest's small team of engineers maintains high accuracy and scalability in their large-scale machine learning system for mining shopping content. Learn about their in-house HTML labeling system for contractor teams and their approach to working with vendor solutions for measuring product content accuracy using anonymous online workers. Gain insights into the systems and processes developed to manage human-in-the-loop workflows with minimal engineering operations time, helping to prevent burnout and improve efficiency in ML production environments.