HyCache - Hybrid Caching for Accelerating DNN Input Preprocessing Pipelines

Learn about HyCache, a novel hybrid caching runtime system designed to accelerate deep neural network input preprocessing pipelines in this 14-minute conference presentation from USENIX ATC '25. Discover how modern GPU advances have shifted the training bottleneck from model computation to CPU-based data loading and preprocessing, creating new performance challenges in end-to-end DNN training. Explore the limitations of existing caching approaches that rely solely on memory or storage and can only cache complete stage outputs from single preprocessing steps. Understand how HyCache overcomes these constraints by enabling partial caching of preprocessed data subsets from multiple intermediate stages across both memory and storage simultaneously. Examine the integer linear programming (ILP) approach used to automatically determine optimal caching strategies that balance recomputation costs against caching benefits without manual intervention. Analyze performance results demonstrating raw pipeline throughput improvements ranging from 1.11× to 10.1× speedups compared to state-of-the-art approaches across various preprocessing pipelines, presented by researchers from the Indian Institute of Science, University of Southern California, and independent research.