Wisedocs' Journey - Rebuilding and Accelerating ML with KubeRay

Learn how to rebuild and accelerate machine learning infrastructure using KubeRay through Wisedocs' real-world transformation journey in this 21-minute conference talk from Ray Summit 2025. Discover the challenges and opportunities involved in redesigning a production-grade ML serving stack as Denys Linkov from Wisedocs details their migration of 10 models to KubeRay, achieving remarkable results including a 50% cost reduction and 10× throughput improvement for both real-time and batch workloads. Explore the architectural decisions that enabled these gains, covering compute orchestration, workload isolation, and scaling strategies that transformed their development-to-production cycle from one month to just two days. Examine the technical architecture and internal abstractions that streamlined deployment patterns using Ray's distributed execution model, and understand the tradeoffs between serving GenAI models versus encoder-based models within an internal Kubernetes environment. Gain practical insights into performance tuning, resource management, and operational complexity considerations while learning how to modernize ML serving stacks with Ray and KubeRay, balance efficiency with reliability, and accelerate production deployment at scale.