Coursera Flash Sale
40% Off Coursera Plus for 3 Months!
Grab it
Learn how to deploy AI agents at scale using Ray Serve as a framework-agnostic foundation from this 11-minute conference talk presented by Apple engineers at Ray Summit 2025. Discover the growing complexity of modern AI agents that span multi-step reasoning, tool use, memory, planning, and dynamic interaction patterns, along with the corresponding deployment challenges in production environments. Explore how traditional serving frameworks with static inference graphs fall short for agentic workloads requiring flexible orchestration and adaptive execution. Understand Apple's approach to leveraging Ray as an Agent Engine by combining Ray Serve's distributed execution model with built-in autoscaling, request routing, and traffic management capabilities. Examine the benefits of framework-agnostic agent deployment that allows integration of agents built using any architecture or library, dynamic scalable execution for complex control flows, high robustness under unpredictable load patterns, and simplified operational workflows. Gain practical insights from real-world deployment experiences at Apple, including patterns that generalize across various agent frameworks and application domains, to build scalable, resilient, and production-ready agent applications using Ray Serve regardless of underlying agent design or development workflow.
