High Performance Computing in Azure - Architecture and Implementation Guide

Learn about High Performance Computing (HPC) implementation in Azure through this comprehensive technical video that explores compute, storage, and orchestration scenarios. Master key HPC patterns including embarrassingly parallel and tightly coupled workloads, understand node resiliency, and examine traditional on-premises architectures. Explore Azure-specific solutions including specialized VM options like H-series and N-series GPU SKUs, InfiniBand networking capabilities, and storage solutions such as Managed Lustre. Gain practical knowledge about Azure-based orchestrators, with detailed coverage of Azure CycleCloud implementation, workspace configuration for Slurm, and Azure Batch services. Discover important considerations about availability zones, Entra ID authentication, and using the CycleCloud CLI for custom templates.

Syllabus

00:00 - Introduction
00:59 - HPC patterns
01:22 - Embarrassingly parallel
03:15 - Tightly coupled
05:30 - Node resiliency importance
06:14 - On-premises HPC architectures
07:01 - HPC end-to-end
10:54 - Azure-based solution
11:36 - HPC virtual machine options
11:56 - InfiniBand secret sauce
13:46 - H-series VMs
17:52 - N-series GPU SKUs
21:26 - Storage solutions
22:35 - Managed Lustre
24:27 - Azure-based orchestrators
25:16 - Azure CycleCloud
26:58 - Note on NOT using Availability Zones
28:29 - Supported orchestrators
30:22 - Entra ID authentication
31:27 - Deploying a scheduler on Azure CycleCloud
32:58 - CycleCloud CLI and custom templates
33:57 - CycleCloud workspace for Slurm
36:24 - Azure Batch
37:31 - Summary
38:44 - Close