Shared-GPU Security Learnings from Fly.io

Learn about the security challenges and solutions for implementing shared GPU access in cloud environments through this 22-minute conference talk by Matthew Braun, Director of Security at Fly.io. Explore the technical difficulties faced when providing secure customer access to NVIDIA GPU hardware, including why existing GPU virtualization technologies proved inadequate for multi-tenant scenarios. Examine the threat landscape associated with shared and virtualized GPU access, focusing on potential attack vectors and security vulnerabilities. Dive deep into the architecture of NVIDIA datacenter-grade GPUs with particular attention to security-relevant subsystems and their implications for shared computing environments. Investigate PCIe functionality from a security perspective, understanding both the threats it presents and available mitigations. Gain insights from real-world security evaluations of hardware and operating system environments, along with practical conclusions and recommendations for organizations considering GPU-as-a-Service offerings. Discover the lessons learned from Fly.io's significant investment in secure GPU compute resources and understand the broader implications for cloud security architecture.