HypeReca - Distributed Heterogeneous In-Memory Embedding Database for Training Recommender Models

Learn about HypeReca, a distributed heterogeneous in-memory embedding database system designed to address the scalability challenges of training deep learning-based recommender models (DLRM) in this 22-minute conference presentation from USENIX ATC '25. Discover how researchers from Tsinghua University tackle the memory and communication bottlenecks that arise when training recommender systems on massive datasets across GPU clusters, where huge embedding tables create significant data management overhead. Explore the key insight that a distributed in-memory key-value database provides the optimal abstraction for serving and maintaining embedding vectors during DLRM training, and understand how HypeReca leverages both GPU and CPU memory to achieve high scalability. Examine the system's innovative pipeline design over decentralized indexing tables, contention-avoiding scheduling for data exchange, and two-fold parallel strategy that ensures consistency across all embedding vectors. Analyze how the system reduces communication overhead through strategic replication of frequently accessed embedding vectors, exploiting sparse access patterns with a performance model. Review the impressive evaluation results showing 2.16-16.8× end-to-end speedup improvements over existing systems like HugeCTR, TorchRec, and TFDE when tested on 32 GPUs with real-world datasets, making this essential viewing for researchers and practitioners working on large-scale machine learning systems and recommender model training infrastructure.