Scalable Parallel Computing Lab, SPCL @ ETH Zurich Courses

Exploring GPU-to-GPU Communication: Insights into Supercomputer Interconnects

Explore the performance characteristics of GPU interconnects in modern supercomputers through comprehensive benchmarking and analysis of intra-node and inter-node communication on systems like Alps, Leonardo, and LUMI.

• YouTube
• 26 minutes
• Self-Paced
• Free Video

Proximity is All You Need - Two Tricks for Chiplet Interconnects

Explore two innovative techniques for optimizing chiplet interconnects: HexaMesh tiling and Folded HexaTorus, designed to maximize bandwidth while adhering to packaging constraints, alongside the RapidChiplet toolchain for design exploration.

• YouTube
• 12 minutes
• Self-Paced
• Free Video

Offloaded MPI Message Matching - An Optimistic Approach

Explore optimistic tag matching for MPI message delivery using SmartNICs to free CPU resources while maintaining performance in high-performance computing environments.

• YouTube
• 23 minutes
• Self-Paced
• Free Video

EDAN - Towards Understanding Memory Parallelism and Latency Sensitivity in HPC

Discover EDAN, a novel tool analyzing memory parallelism and latency sensitivity in HPC applications through execution DAG generation from runtime instruction traces.

• YouTube
• 23 minutes
• Self-Paced
• Free Video

ARMing GPUs for Impactful Science with the GH200 Superchip and True Exascale

Explore NVIDIA's GH200 Superchip architecture, analyzing its complex memory system and demonstrating revolutionary large-scale scientific computing applications.

• YouTube
• 21 minutes
• Self-Paced
• Free Video

Merging and MoErging for Compositional Generalization

Explore Colin Raffel's research on model merging and MoErging techniques for compositional generalization, examining why merging fails and how routing between individual-task models offers a promising alternative approach.

• YouTube
• 56 minutes
• Self-Paced
• Free Video

The Evolution of Accelerator-Centric GPU Services - Past, Present and Future

Explore the evolution of GPU systems from gaming processors to AI powerhouses, focusing on accelerator-centric designs that enable direct GPU access to storage and network services without CPU intervention.

• YouTube
• 1 hour 1 minute
• Self-Paced
• Free Video

Deep Learning for Weather Prediction - Advances and Future Directions

Explore how deep learning models are revolutionizing weather prediction, surpassing traditional numerical methods with improved accuracy and longer forecast capabilities.

• YouTube
• 1 hour 10 minutes
• Self-Paced
• Free Video

From Large Language Models to Reasoning Language Models - Three Eras in the Age of Computation

Explore the evolution of LLMs through computational advances, optimization breakthroughs, and the emergence of reasoning models, including insights into Ultra Ethernet and future AI infrastructure.

• YouTube
• 1 hour 1 minute
• Self-Paced
• Free Video

Swing: Short-cutting Rings for Higher Bandwidth Allreduce

Optimize allreduce performance on torus networks with Swing algorithm. Improve bandwidth by reducing distance between communicating nodes, outperforming existing methods by up to 3x.

• YouTube
• 26 minutes
• Self-Paced
• Free Video

HOT: Higher-Order Dynamic Graph Representation Learning with Efficient Transformers

Explore higher-order dynamic graph representation learning with efficient Transformers for improved link prediction in rapidly changing graphs. Learn about HOT model's architecture and performance.

• YouTube
• 20 minutes
• Self-Paced
• Free Video

VENOM: A Vectorized N:M Format for Unleashing the Power of Sparse Tensor Cores

Explore VENOM, a vectorized N:M format for sparse tensor cores. Learn about high-performance sparse libraries and second-order pruning techniques for efficient deep learning model sparsification.

• YouTube
• 25 minutes
• Self-Paced
• Free Video

Motif Prediction with Graph Neural Networks

Explore advanced graph mining techniques using Graph Neural Networks for motif prediction, enhancing link prediction capabilities and enabling higher-order network analysis.

• YouTube
• 23 minutes
• Self-Paced
• Free Video

The Digital Revolution of Earth System Modelling

Explore Earth system modeling's evolution: quiet, digital, and machine learning revolutions. Discover past developments, current challenges, and future prospects in weather prediction and climate simulation.

• YouTube
• 1 hour 12 minutes
• Self-Paced
• Free Video

Capturing Computation with Algorithmic Alignment

Explore algorithmic alignment in neural networks, its implications for architecture design, and its potential for building future intelligent systems.

• YouTube
• 1 hour 1 minute
• Self-Paced
• Free Video