Distributed Training Courses and Certifications

USENIX

Holmes - Localizing Irregularities in LLM Training with Mega-scale GPU Clusters

Discover Holmes, a system that localizes training irregularities in large-scale GPU clusters for LLMs, achieving 97.21% accuracy and 6.52× speedup over traditional methods.

• YouTube
• 16 minutes
• Self-Paced
• Free Video

USENIX

Primus - Unified Training System for Large-Scale Deep Learning Recommendation Models

Discover Primus, ByteDance's unified training system for large-scale deep learning recommendation models, featuring dynamic scaling, data orchestration, and hybrid training paradigms.

• YouTube
• 17 minutes
• Self-Paced
• Free Video

TensorFlow

Distributed TensorFlow Training - Google I/O 2018

Learn to scale TensorFlow training across multiple GPUs and machines efficiently. Explore distribution strategies, performance optimization techniques, and practical implementation using Google Cloud.

• YouTube
• 35 minutes
• Self-Paced
• Free Video

USENIX

TopoOpt - Co-optimizing Network Topology and Parallelization Strategy for Distributed Training Jobs

Novel direct-connect fabric optimizing distributed DNN training across computation, communication, and network topology. Reduces training time by up to 3.4x compared to similar-cost Fat-Tree interconnects.

• YouTube
• 17 minutes
• Self-Paced
• Conference Talk

Anyscale

Scaling Computer Vision Models with Ray - A Cost-Effective and Efficient Distributed Training Framework

Scale computer vision models efficiently using Ray's distributed training framework. Compare performance with Kubeflow, leveraging cost-effective S3 storage and optimized data processing for improved speed and GPU utilization.

• YouTube
• 35 minutes
• Self-Paced
• Free Video

MLOps World: Machine Learning in Production

Supercharging Recommender Systems - Unleashing the Power of Distributed Model Training

Explore distributed model training techniques to enhance recommender systems, boosting performance and scalability for production environments.

• YouTube
• 22 minutes
• Self-Paced
• Free Video

Azure AI Training and Fine-tuning Custom LLMs with Distributed Training

Master distributed training techniques for custom LLMs using Azure AI infrastructure, focusing on performance optimization, scalability, and efficient deployment with ND H100 and H200 series.

• YouTube
• 44 minutes
• Self-Paced
• Free Video

Anyscale

Spotify's Approach to Distributed LLM Training with Ray on GKE

Dive into Spotify's innovative approach to distributed LLM training using Ray on GKE. Learn about efficient training of 70B+ parameter models, resource management, and performance optimization techniques for scalable ML infrastructure.

• YouTube
• 31 minutes
• Self-Paced
• Free Video

Using Training-Operator to Schedule Distributed Edge-Cloud Collaborative AI Applications

Explore how to integrate KubeFlow training-operator with KubeEdge's Sedna framework to enable distributed edge-cloud collaborative AI applications, optimizing resource allocation and enhancing efficiency.

• YouTube
• 20 minutes
• Self-Paced
• Free Video

Empowering ML Workloads With Kubeflow: JAX Distributed Training and LLM Hyperparameter Optimization

Discover how Kubeflow innovations enhance distributed JAX training on Kubernetes and automate hyperparameter optimization for LLMs, making complex ML workloads more efficient and establishing Kubeflow as a powerful platform for modern AI development.

• YouTube
• 20 minutes
• Self-Paced
• Free Video

International Centre for Theoretical Sciences

Introduction to Machine Learning, Deep Learning and Distributed Training

Discover machine learning fundamentals, deep learning techniques, and distributed training methods for analyzing fast radio transients and astronomical big data applications.

• YouTube
• 1 hour 37 minutes
• Self-Paced
• Free Video

Anyscale

Distributed ML Training with KubeRay at Robinhood

Discover how Robinhood scaled ML training with KubeRay, covering architectural decisions, platform abstractions, and real-world integration strategies for distributed training at scale.

• YouTube
• 32 minutes
• Self-Paced
• Free Video

Unlocking Scalable Distributed Training With Arrow Data Cache on Kubernetes

Discover how to eliminate data bottlenecks in AI training using Arrow-based caching on Kubernetes for scalable, cloud-native machine learning workloads with improved GPU utilization.

• YouTube
• 26 minutes
• Self-Paced
• Free Video

Coursera

GPU Clusters & Containers

Unlock distributed AI training and production deployment using cloud GPU clusters and containerization strategies for scalable machine learning systems.

• Coursera
• 2 hours 5 minutes
• Self-Paced
• Paid Course

Coursera Credential

Blueprint to Bytecode: Architecting Scalable AI Systems

Master the full AI systems lifecycle—from data pipelines and GPU clusters to cloud deployment—using Kubernetes, Docker, SageMaker, and MLOps best practices to build scalable, production-ready enterprise AI solutions.

• Coursera
• 4 weeks, 10 hours/week
• Self-Paced
• Paid Course
• 10 courses