Orchestrating AI Agents to Structure the World's Product Knowledge

Learn how to scale product taxonomy management across millions of diverse items using orchestrated AI agents in this conference talk from MLOps World. Discover Shopify's innovative approach to solving the impossible challenge of maintaining product classifications spanning everything from guitar picks to industrial sensors when no human team could possess expertise across every domain. Explore the architectural patterns behind parallel, specialized AI agents that collaborate to analyze, improve, and validate taxonomy changes while maintaining quality, consistency, and auditability at scale. Understand how LLM-as-Judge systems revolutionized quality control processes and examine the critical production lessons learned from deploying multi-agent orchestration systems. Gain insights into decomposing complex classification tasks into manageable, domain-specific agent workflows, implementing scalable quality assurance without full human review, and building trust through transparency, reasoning, and comprehensive audit trails in agentic systems. Master the core principle that effective scaling amplifies human experts across domains they cannot cover alone rather than replacing them entirely, with practical guidance on prompt optimization, fail-safe recovery patterns, and production deployment strategies for AI agent orchestration.