XRT - An Accelerator-Aware Runtime for Accelerated Chip Multiprocessors

Learn about XRT, an innovative accelerator-aware runtime system designed for modern chip multiprocessors in this 15-minute conference presentation from USENIX ATC '25. Discover how datacenter applications increasingly rely on specialized accelerators to execute common functions with superior performance and energy efficiency, while simultaneously requiring microsecond-scale response times and low tail latency. Explore the challenges faced by current runtime systems when dealing with on-chip accelerators in chip multiprocessors (CMPs), where despite hardware support enabling fast communication between general-purpose cores, direct accelerator access to memory subsystems, and scalable resource sharing, runtime-level limitations prevent efficient utilization at scale. Understand how existing runtimes prove unsuitable for accelerator-heavy applications, sometimes yielding performance worse than systems without accelerators entirely. Examine the design and implementation of XRT, a specialized runtime solution engineered to scale effectively across many-core, many-accelerator CPU architectures. Analyze performance results demonstrating XRT's capability to achieve up to 3.2× higher throughput under service level objectives compared to unoptimized runtimes, while consistently avoiding performance degradation relative to systems executing all request processing on general-purpose cores. Gain insights into the architectural considerations and optimization strategies that enable efficient accelerator utilization in modern datacenter environments through this research presented by Neel Patel and Mohammad Alian from Cornell University.