3L-Cache: Low Overhead and Precise Learning-based Eviction Policy for Caches

This conference talk from USENIX FAST '25 presents 3L-Cache, an innovative object-level learning policy for cache systems designed to achieve low computation overhead while delivering the lowest object and byte miss ratios among learning-based policies. Discover how researchers from Beijing University of Technology and Microsoft Research developed two key advancements to reduce overhead: an efficient training data collection scheme that filters unnecessary historical cache requests and dynamically adjusts training frequency, and a low-overhead eviction method combining bidirectional sampling to prioritize unpopular objects with an efficient eviction strategy. Learn about their parameter auto-tuning method that enhances adaptability across different traces. The presentation shares evaluation results from 4,855 traces showing that 3L-Cache reduces average CPU overhead by 60.9% compared to HALP and 94.9% compared to LRB, while incurring only 6.4× the average overhead of LRU for small cache sizes and 3.4× for large cache sizes—all while achieving the best byte or object miss ratio among twelve state-of-the-art policies.