Demystifying NCCL - An In-Depth Analysis of GPU Communication Protocols and Algorithms
HOTI - Hot Interconnects Symposium via YouTube
Learn AI, Data Science & Business — Earn Certificates That Get You Hired
Build AI Apps with Azure, Copilot, and Generative AI — Microsoft Certified
Overview
AI, Data Science & Cloud Certificates from Google, IBM & Meta — 40% Off
One plan covers every Professional Certificate on Coursera. 40% off Coursera Plus Annual.
Unlock All Certificates
Explore the intricacies of NVIDIA Collective Communication Library (NCCL) through this 29-minute conference talk from the Hot Interconnects Symposium. Delve into comprehensive analysis of GPU communication protocols and algorithms as presented by researchers from ETH Zurich, NVIDIA, and Broadcom. Examine the underlying mechanisms that enable efficient multi-GPU communication, understand the design principles behind NCCL's collective operations, and gain insights into the performance optimization strategies used in modern GPU clusters. Learn about the communication patterns, network topologies, and algorithmic approaches that make large-scale distributed GPU computing possible, with detailed explanations of how NCCL handles bandwidth optimization, latency reduction, and scalability challenges in high-performance computing environments.
Syllabus
Demystifying NCCL An In depth Analysis of GPU Communication Protocols and Algorithms - Zhiyi Hu
Taught by
HOTI - Hot Interconnects Symposium