Coursera Flash Sale
40% Off Coursera Plus for 3 Months!
Grab it
Learn how Adobe accelerated large-scale Generative AI training for Adobe Firefly through JIT-Embedding (Just-in-Time Embedding) in this 31-minute conference talk from Ray Summit 2025. Discover how Haoran Cai and Baqiao Liu from Adobe developed a novel Ray Serve-powered architecture that decouples embedding computation from model training to dramatically improve scalability, performance, and cost efficiency. Explore the core bottlenecks in foundation diffusion model training for image and video generation, including slow on-the-fly embedding computation (VAE, CLIP, T5), high costs of offline embedding precomputation, and severe GPU memory constraints when training high-resolution or large-scale models. Understand the major components and innovations of JIT-Embedding, including the JIT Service via Ray Serve that wraps embedding computation as an on-demand, autoscaled service deployed on underutilized lower-tier GPUs, freeing H100s for training while reducing GPU memory pressure. Examine the JIT Client integrated with the Dataloader that uses multiprocessing and prefetching to overlap embedding requests with training, hiding latency and maximizing end-to-end GPU utilization. Learn about the high-throughput serialization/deserialization system using a custom Rust + Python library that compresses multimodal data to speed up client-server communication. Discover advanced performance optimizations including Ray Serve's dashboards plus Adobe's custom profiling and load-testing tools that enable dynamic batching, autoscaling, in-place updates, client-side load balancing, overlapping CPU/GPU execution, optimized codecs, and shared GPU usage across models. Explore the JIT Cache system that automatically stores computed embeddings for reuse in future training jobs, further reducing compute time and cost. Review end-to-end experimental results demonstrating how JIT-Embedding significantly improved scalability, enabled higher-resolution Firefly model training, and delivered substantial performance gains and cost reductions that contributed to the public release of the Firefly Video Model. Gain insights into future directions, lessons learned from building on Ray Serve, and Adobe's plan to open source the entire JIT-Embedding stack, including services, clients, and the serialization library.
