Private Retrieval-Augmented Generation: Secure Search Systems for LLMs

Watch a technical lecture from UC Berkeley's Raluca Popa exploring Compass, a novel semantic search system designed for encrypted data in Retrieval-Augmented Generation (RAG) applications. Delve into how Compass achieves security for sensitive RAG databases and queries while maintaining high semantic accuracy comparable to unencrypted algorithms. Learn about the system's innovative combination of cryptographic and machine learning techniques, including a cryptography-friendly algorithm for traversing Hierarchical Navigable Small Worlds graphs integrated with Oblivious RAM. Understand how this approach enables practical search performance with latencies of just a few seconds, significantly outperforming existing encrypted embedding search methods while ensuring complete protection of data, queries, and search results from compromised servers.