Distributed Deep Learning Courses and Certifications

Efficient AI Model Training with PyTorch

106 ratings at DataCamp

Distributed training is an essential skill in large-scale machine learning, helping you to reduce the time required to train large language models with trillions of parameters.

• DataCamp
• 4 hours
• Self-Paced
• Free Trial Available

Anyscale

Ray Train: A Production-Ready Library for Distributed Deep Learning

Explore Ray Train's architecture for efficient, cost-effective distributed deep learning. Learn about resource scheduling, API simplicity, and exclusive features for LLM training, including Distributed Checkpointing.

• YouTube
• 32 minutes
• Self-Paced
• Free Video

Running Distributed TensorFlow using Vertex AI

This is a self-paced lab that takes place in the Google Cloud console. In this lab, you will use TensorFlow's distribution strategies and the Vertex AI platform to train and deploy a custom TensorFlow image classification model to classify an image class…

• Coursera
• 1 hour 30 minutes
• Self-Paced
• Paid Course

Packt

Train Large Language Models Faster - Parallelism Deep Dive

Implement data, model, pipeline, and tensor parallelism for LLM training in PyTorch and DeepSpeed, run multi-GPU jobs on Runpod.io, and apply checkpointing and fault tolerance.

• Coursera
• 16 weeks, 1 hour a week
• Self-Paced
• Paid Course

Distributed Training & Advanced Application

Scale transformers across multi-node GPU clusters with DeepSpeed ZeRO-3 sharding, activation checkpointing, CPU offloading, and Azure ML remote-job submission.

• Coursera
• 1 hour
• Self-Paced
• Paid Course

Using GPUs to Scale and Speed-up Deep Learning

1 rating

Training complex deep learning models with large datasets takes along time. In this course, you will learn how to use accelerated GPU hardware to overcome the scalability problem in deep learning.

• edX
• 5 weeks, 2-4 hours a week
• Self-Paced
• Free Certificate

Let's Build Pipeline Parallelism from Scratch - Tutorial

Build pipeline parallelism from scratch, progressing from manual model partitioning and distributed communication to GPipe micro-batching and 1F1B scheduling.

• freeCodeCamp
• 3 hours 22 minutes
• Self-Paced
• Free Video

Aleksa Gordić - The AI Epiphany

Ultimate Guide to Scaling ML Models - Megatron-LM - ZeRO - DeepSpeed - Mixed Precision

Learn how data, tensor, pipeline, and 3D parallelism, mixed precision, activation checkpointing, and ZeRO scale models to trillions of parameters.

• YouTube
• 1 hour 23 minutes
• Self-Paced
• Free Video

Board Infinity

Deep Learning: Advanced Backbones and Efficient GPU Training

Build and fine-tune ConvNeXt and Vision Transformer backbones with PyTorch Lightning and timm, applying RMSNorm, SwiGLU, RoPE, mixed precision, DDP/FSDP, and W&B tracking.

• Coursera
• 4 weeks, 5 hours a week
• Self-Paced
• Paid Course

Edureka

Generative AI Models and GPU Systems

Explore autoencoders, VAEs, GAN dynamics, DDPM and latent diffusion, then scale training with mixed precision, distributed data parallel, and production GPU deployment.

• Coursera
• 4 weeks, 3 hours a week
• Self-Paced
• Paid Course

Packt

Accelerate Model Training with PyTorch 2.X

Speed up deep learning training in PyTorch 2.X: compile models, apply mixed precision and pruning, build efficient data pipelines, and distribute across CPUs, GPUs, and machines.

• Coursera
• 11 weeks, 1 hour a week
• Self-Paced
• Paid Course

Career Certificate

Microsoft Deep Learning Engineering with Azure

Engineer the full deep learning lifecycle on Azure: implement CNNs, RNNs, and Transformers; fine-tune models with LoRA and QLoRA; scale training with DeepSpeed ZeRO; deploy containerized models.

• Coursera
• Self-Paced
• Paid Course
• 3 courses

Anyscale

Distributed Mesh Processing and Geometric Deep Learning for Wind Prediction

Discover how Ray's distributed computing powers real-time wind prediction in urban design through mesh processing, geometric deep learning, and CFD data analysis for architectural innovation.

• YouTube
• 14 minutes
• Self-Paced
• Free Video

Running Distributed TensorFlow using Vertex AI

In this lab, you will use TensorFlow's distribution strategies and the Vertex AI platform to train and deploy a custom TensorFlow image classification model to classify an image classification dataset.

• Google Skills
• 1-2 hours
• Self-Paced
• Paid Course

Bagua - Lightweight Distributed Learning on Kubernetes

Explore Bagua, a high-performance distributed deep learning project for Kubernetes, offering innovative communication algorithms and seamless integration for efficient training across various scenarios.

• YouTube
• 36 minutes
• Self-Paced
• Conference Talk