Orchestrating AI Agents to Structure the World's Product Knowledge

Learn how to orchestrate specialized AI agents to manage massive product taxonomies spanning millions of diverse items in this conference talk from MLOps World. Discover Shopify's innovative approach to solving the impossible challenge of maintaining product classification across domains where no single human team could hold comprehensive expertise, from guitar picks to industrial sensors. Explore the architectural patterns behind parallel AI agent collaboration that analyzes, improves, and validates taxonomy changes while maintaining quality, consistency, and auditability at scale. Understand how LLM-as-Judge systems revolutionized quality control by enabling scalable quality assurance without requiring full human review of every decision. Gain insights into critical production lessons including managing prompt overload, implementing fail-safe recovery patterns, and building trust through transparency and audit trails in agentic workflows. Master the techniques for decomposing complex classification tasks into manageable, domain-specific agent responsibilities and learn why the key to scaling isn't replacing human experts but amplifying their capabilities across domains they cannot cover alone. Examine real-world deployment challenges and solutions from Shopify's production environment, including how to maintain reasoning transparency and create reliable multi-agent orchestration systems that can handle the complexity of global e-commerce product categorization.