Layering Every Technique in RAG - One Query at a Time

Learn to build sophisticated Retrieval-Augmented Generation (RAG) systems through an incremental, technique-by-technique approach in this 20-minute conference talk. Start with basic in-memory embeddings and relevance ranking, then progressively layer advanced techniques to reach planet-scale search capabilities handling 160,000 queries per second. Explore the evolution from simple BM25 term-based retrieval to complex systems combining 70+ corpus mix of tokens, embeddings, and knowledge graphs with joint retrieval, custom ranking, and LLM processing. Discover why certain queries like "falafel" present unique search challenges, understand when document chunking can be counterproductive, and learn to identify scenarios where simple BM25 suffices versus when problems are better delegated to LLMs or user experience design. Master the quality engineering mindset essential for building robust search systems, covering vector search with relevance embeddings, cross-encoder re-rankers, custom embeddings, domain-specific ranking signals, user preference integration, query orchestration with fan-out strategies, supplementary retrieval methods, distillation techniques, and graceful degradation approaches. Gain insights from Google Search's core AI and NLU systems development experience, learning practical strategies for handling increasingly complex RAG queries while understanding the limitations and capabilities unlocked by each layered technique.