Distributed Training Courses and Certifications

Decision-Making in Dynamic Environments

Master multi-agent AI systems through game theory, distributed training, and communication protocols to deploy scalable solutions in dynamic real-world environments.

• Coursera
• 2 hours 41 minutes
• On-Demand
• Paid Course

Coursera

GPU Clusters & Containers

Unlock distributed AI training and production deployment using cloud GPU clusters and containerization strategies for scalable machine learning systems.

• Coursera
• 2 hours 5 minutes
• On-Demand
• Paid Course

AI Infrastructure: Deployment Types

Explore Google Cloud's AI infrastructure deployment strategies, from Compute Engine to Kubernetes, with hands-on cluster creation and inference optimization techniques.

• Coursera
• 5 hours 11 minutes
• On-Demand
• Paid Course

Packt

Train Large Language Models Faster - Parallelism Deep Dive

Master parallelism strategies to accelerate large language model training using PyTorch and DeepSpeed across multi-GPU systems with fault tolerance techniques.

• Coursera
• 13 hours 44 minutes
• On-Demand
• Paid Course

Packt

Accelerate Model Training with PyTorch 2.X

Master PyTorch 2.X optimization techniques to dramatically accelerate model training through compilation, mixed precision, distributed strategies, and efficient data pipelines.

• Coursera
• 8 hours 10 minutes
• On-Demand
• Paid Course

Coursera

Deploy and Optimize Cloud AI Architectures

Master scalable ML deployment on Amazon SageMaker with distributed training, Spot Instances, autoscaling, and cost optimization through GPU monitoring and CloudWatch metrics.

• Coursera
• 1 hour 48 minutes
• On-Demand
• Paid Course

Accelerate Model Training with a High-Performance Distributed AI/ML Stack for the Cloud

Learn to accelerate AI/ML model training using Ray and Open Data Hub for efficient resource management and distributed computing in cloud environments, enhancing scalability and performance.

• YouTube
• 45 minutes
• On-Demand
• Free Video

USENIX

SHADE - Enable Fundamental Cacheability for Distributed Deep Learning Training

Innovative caching system for distributed deep learning training, improving performance by prioritizing important data samples and enhancing cache hit ratios in I/O-intensive workloads.

• YouTube
• 18 minutes
• On-Demand
• Conference Talk

USENIX

Dorylus - Affordable, Scalable, and Accurate GNN Training with Distributed CPU Servers and Serverless Threads

Distributed system for training Graph Neural Networks using serverless computing, offering affordable scalability for billion-edge graphs with improved performance-per-dollar compared to GPU servers.

• YouTube
• 15 minutes
• On-Demand
• Conference Talk

USENIX

KungFu - Making Training in Distributed Machine Learning Adaptive

KungFu: adaptive distributed machine learning library for TensorFlow. Enables real-time parameter tuning using high-level policies, efficient monitoring, and asynchronous communication for improved training performance.

• YouTube
• 20 minutes
• On-Demand
• Conference Talk

USENIX

Accelerating Distributed MoE Training and Inference with Lina

Explore Lina, an innovative system for accelerating distributed Mixture of Experts training and inference, addressing all-to-all communication bottlenecks in large-scale AI models.

• YouTube
• 20 minutes
• On-Demand
• Conference Talk

High Performance Networking for Distributed DL Training in Production K8s

Explore high-performance networking for distributed deep learning on Kubernetes, covering 800 GPU cluster design, RoCE fabric, and optimizations for NCCL rings and GPU utilization.

• YouTube
• 25 minutes
• On-Demand
• Conference Talk

USENIX

MAST: Global Scheduling of ML Training across Geo-Distributed Datacenters at Hyperscale

Intelligent global scheduling system for ML training workloads across datacenters, optimizing resource utilization and eliminating regional overload through innovative design principles.

• YouTube
• 16 minutes
• On-Demand
• Free Video

MLOps World: Machine Learning in Production

Supercharging Recommender Systems - Unleashing the Power of Distributed Model Training

Explore distributed model training techniques to enhance recommender systems, boosting performance and scalability for production environments.

• YouTube
• 22 minutes
• On-Demand
• Free Video

MLOps World: Machine Learning in Production

Scale and Accelerate Distributed Model Training in Kubernetes Clusters

Orchestrate and accelerate distributed deep learning workloads across multiple GPUs and nodes using Kubernetes, Kubeflow PytorchJob, RDMA, RoCE, and SRIOV for near-linear performance scaling.

• YouTube
• 49 minutes
• On-Demand
• Free Video