Anyscale Courses

Optimizing vLLM Performance Through Quantization - Model Compression Techniques

Discover practical techniques for model quantization in vLLM deployments, exploring FP8, INT8, and INT4 methods to enhance performance and reduce memory usage while maintaining efficiency.

• YouTube
• 38 minutes
• Self-Paced
• Free Video

Scaling Video Ad Classification with Ray for Multi-Million Category Processing

Discover how to scale video ad classification to millions of categories using Ray and LLMs, featuring real-world implementation strategies from Vivvix and Databricks' successful collaboration.

• YouTube
• 31 minutes
• Self-Paced
• Free Video

Building Intelligent AI Infrastructure with ORI - Dynamic Query Routing and Model Management

Discover how ORI dynamically routes AI queries to optimal language models, featuring chain atomic queries, benchmark matching, and ethical frameworks integrated with Ray Serve and vLLM for high-speed inference.

• YouTube
• 22 minutes
• Self-Paced
• Free Video

The Evolution of Multi-GPU Inference in vLLM

Dive into advanced parallelism strategies for distributed LLM inference, exploring tensor, pipeline, and expert parallelism techniques through vLLM's architecture and optimization approaches.

• YouTube
• 31 minutes
• Self-Paced
• Free Video

Intelligent Data Classification with Ray and vLLM at Apple

Discover how Apple leverages Ray and vLLM for AI-powered data classification, addressing privacy, documentation quality, and business context challenges while automating data governance processes.

• YouTube
• 25 minutes
• Self-Paced
• Free Video

How Roblox Scaled Machine Learning by Leveraging Ray for Efficient Batch Inference

Discover how Roblox leverages Ray to scale machine learning infrastructure, focusing on multimodal language models integration, batch inference optimization, and practical implementation strategies.

• YouTube
• 29 minutes
• Self-Paced
• Free Video

Scaling Ray Train to 10,000 Kubernetes Nodes on GKE

Discover how to scale distributed machine learning workloads across 10,000 Kubernetes nodes using Ray Train and GKE, featuring architectural insights and practical solutions for massive AI model training.

• YouTube
• 35 minutes
• Self-Paced
• Free Video

Accelerating Autonomous Driving Development with Ray

Discover how Zoox leverages Ray's capabilities to enhance machine learning pipelines for autonomous vehicles, from model training to real-time deployment and scaling on Amazon EKS.

• YouTube
• 22 minutes
• Self-Paced
• Free Video

Chatbot Arena: Evaluating AI Models Through Human Preferences

Explore UC Berkeley's Chatbot Arena platform and its innovative approach to evaluating AI models through human preferences, pairwise comparisons, and crowdsourced data from 1.7M+ user votes.

• YouTube
• 32 minutes
• Self-Paced
• Free Video

Dynamic Scheduling for Large Language Model Serving

Discover how Llumnix optimizes large language model serving through dynamic request scheduling, cross-instance defragmentation, and innovative KV cache migration techniques for improved performance.

• YouTube
• 32 minutes
• Self-Paced
• Free Video

Building an Agentic Framework for Generative AI Applications

Discover how to build an agentic framework for GenAI applications, focusing on RAG-centric approaches, multilingual queries, and real-time data processing for enhanced customer assistance.

• YouTube
• 32 minutes
• Self-Paced
• Free Video

Streamlining AI Workflows with Apache Airflow and Ray

Discover how to build a unified AI infrastructure by integrating Apache Airflow and Ray for seamless orchestration of data processing, model training, and inference workflows, including LLM fine-tuning and LoRA deployments.

• YouTube
• 21 minutes
• Self-Paced
• Free Video

Overview of CloudKitchens's Ray-Powered Machine Learning Platform

Discover how CloudKitchens leverages Ray and Kubernetes to build a state-of-the-art ML platform, featuring seamless integration of Metaflow, Daft, and Argo Workflows for enhanced development efficiency.

• YouTube
• 32 minutes
• Self-Paced
• Free Video

How the AI Institute is Revolutionizing Robotics Machine Learning Training

Discover how Boston Dynamics AI Institute revolutionizes robotics ML with an innovative training platform that integrates multimodal data, extensive simulations, and scalable infrastructure using Ray.

• YouTube
• 35 minutes
• Self-Paced
• Free Video

Evolving Ray Core - Scalability, Reliability, and Compiled Graphs

Discover the latest advancements in Ray Core's architecture, including streaming generators, weekly releases, and the new Compiled Graphs backend for enhanced performance in accelerator-driven computing environments.

• YouTube
• 37 minutes
• Self-Paced
• Free Video