Parallel Paths to High-Bandwidth Memory for ML/AI - Specific Purpose Memory vs Driver-Managed Approaches

Explore two distinct approaches for integrating high-bandwidth memory (HBM) into machine learning and AI systems in this 23-minute conference talk from the Linux Plumbers Conference. Examine the critical role of HBM in modern ML/AI workloads and understand how platform firmware and operating system drivers must evolve to effectively present these new memory types to applications. Compare Specific Purpose Memory (SPM) as defined in UEFI 2.8 with driver-managed approaches, analyzing how each paradigm handles HBM integration on GPU accelerators like AMD's MI200 series and NVIDIA's Grace Hopper/Blackwell architectures. Learn about firmware tagging mechanisms, kernel interpretation processes, and driver claiming strategies while understanding the implementation details of Intel's DAX integration and its impact on the SPM approach. Discover how NVIDIA's CUDA driver uses hot-plug mechanisms to expose HBM as system RAM through driver-managed NUMA domains, contrasting this with AMD's SPM-based approach for heterogeneous memory management. Gain insights into kernel configurations, memory types in Linux, and the trade-offs between firmware dependency, allocator compatibility, and runtime configurability in next-generation AI memory systems.