Scalable Parallel Computing Lab, SPCL @ ETH Zurich Courses

Finding Relevant Items Using Approximate Nearest Neighbor Search Algorithms

Explore algorithms for nearest neighbor search, from KD-Trees to modern approaches like HNSW and IVF-PQ, understanding their applications and performance tradeoffs.

• YouTube
• 22 minutes
• Self-Paced
• Free Video

Exascale Cloud Computing - A Foggy Tale of Networks, AI, Containers, and Ultra Ethernet

Dive into advanced cloud computing concepts, exploring datacenter networks, AI acceleration services, and Ultra Ethernet through real-world performance analyses and industry developments.

• YouTube
• 33 minutes
• Self-Paced
• Free Video

Hardware-aware Algorithms for Language Modeling - FlashAttention and Mamba Architectures

Delve into advanced hardware optimizations for language models, exploring FlashAttention-3's improvements and Mamba's efficient architecture for enhanced performance and linear scaling capabilities.

• YouTube
• 1 hour 5 minutes
• Self-Paced
• Free Video

Chameleon - Heterogeneous and Disaggregated Accelerator System for Retrieval-Augmented Language Models

Discover Chameleon, a heterogeneous accelerator system combining LLM and vector search accelerators for efficient Retrieval-Augmented Language Models with up to 3.18× throughput improvements.

• YouTube
• 25 minutes
• Self-Paced
• Free Video

LLAMP - Assessing Network Latency Sensitivity Tolerance of HPC Applications with Linear Programming

Discover LLAMP, a novel toolchain using linear programming to assess HPC applications' network latency tolerance efficiently without specialized hardware or simulators.

• YouTube
• 30 minutes
• Self-Paced
• Free Video

ARMing GPUs for Impactful Science with the GH200 Superchip and True Exascale

Explore NVIDIA's GH200 Superchip architecture, analyzing its complex memory system and demonstrating revolutionary large-scale scientific computing applications.

• YouTube
• 21 minutes
• Self-Paced
• Free Video

Offloaded MPI Message Matching - An Optimistic Approach

Explore optimistic tag matching for MPI message delivery using SmartNICs to free CPU resources while maintaining performance in high-performance computing environments.

• YouTube
• 23 minutes
• Self-Paced
• Free Video

Measurement and Analysis of Application Performance on Exascale GPU-accelerated Systems

Discover how HPCToolkit enables instruction-level performance measurement and analysis on GPU-accelerated exascale systems, with techniques for efficient data handling, parallel analysis, and visualization of terabytes of performance data.

• YouTube
• 1 hour 14 minutes
• Self-Paced
• Free Video

Data Selection - Data Challenges when Training Generative Models

Explore strategic data selection techniques for efficient generative AI model training, covering filtering methods for pre-training and optimal transport approaches for fine-tuning that reduce data requirements while maintaining performance.

• YouTube
• 1 hour
• Self-Paced
• Free Video

All Models Are Wrong, Some Are Useful: Model Selection with Limited Labels

Discover MODEL SELECTOR, a framework that efficiently identifies the best pretrained classifier using minimal labeled data, reducing labeling costs by up to 94.15% across 1,500+ models on 16 datasets.

• YouTube
• 25 minutes
• Self-Paced
• Free Video

RapidChiplet - How to Explore the Design Space of Inter-Chiplet Interconnects

Explore RapidChiplet, a fast toolchain for designing inter-chiplet interconnects with 427x-137,682x speedup over cycle-level simulations while maintaining high accuracy for chiplet architecture optimization.

• YouTube
• 12 minutes
• Self-Paced
• Free Video

Proximity is All You Need - Two Tricks for Chiplet Interconnects

Explore two innovative techniques for optimizing chiplet interconnects: HexaMesh tiling and Folded HexaTorus, designed to maximize bandwidth while adhering to packaging constraints, alongside the RapidChiplet toolchain for design exploration.

• YouTube
• 12 minutes
• Self-Paced
• Free Video

High Performance Unstructured SpMM Computation Using Tensor Cores

Discover how to accelerate sparse matrix-matrix multiplication using Tensor Cores with SMaT, a novel library that outperforms existing solutions by up to 125x for unstructured sparse matrices on modern GPUs.

• YouTube
• 31 minutes
• Self-Paced
• Free Video

Exploring GPU-to-GPU Communication: Insights into Supercomputer Interconnects

Explore the performance characteristics of GPU interconnects in modern supercomputers through comprehensive benchmarking and analysis of intra-node and inter-node communication on systems like Alps, Leonardo, and LUMI.

• YouTube
• 26 minutes
• Self-Paced
• Free Video

MARLIN: Mixed-Precision Auto-Regressive Parallel Inference on Large Language Models

Discover how MARLIN optimizes LLM inference through mixed-precision techniques, enabling significant speedups for batched workloads while maintaining accuracy through innovative GPU kernel design and quantization support.

• YouTube
• 13 minutes
• Self-Paced
• Free Video