Ray at Scale - Apple's Approach to Elastic GPU Management

Explore Apple's innovative approach to elastic GPU management for scaling AI/ML workloads in this 31-minute conference talk from Ray Summit 2024. Discover how Weiwei Yang and Abin Shahab tackle common challenges like GPU fragmentation, low utilization, and compromised SLAs by building a multi-tenancy ready platform based on Ray. Learn about their queuing and GPU quota management system powered by Apache YuniKorn, and gain insights into advanced techniques for achieving resource fairness, GPU preemption, and gang scheduling across diverse Ray workloads. Gain valuable knowledge for optimizing GPU resource management and enhancing the scalability and efficiency of AI/ML operations in large-scale environments.