The Truth About RAG and vLLM - Why Your Multimodal System Fails at Scale

Discover the critical optimizations needed to build production-ready Retrieval-Augmented Generation (RAG) and self-hosted multimodal systems in this 47-minute technical conference talk. Learn why most RAG implementations fail at scale and master the essential decisions around embedding models, vector indexing, and inference optimization that separate successful deployments from failed experiments. Explore the fundamentals of vector search and understand the tradeoffs between different indexing approaches including FLAT, IVF, and HNSW algorithms. Examine the speed versus accuracy versus cost matrix that governs index selection and discover why choosing the wrong embedding models can doom your project from the start. Dive deep into the complete RAG pipeline from unstructured data processing to retrieval, and understand why proper evaluation frameworks are essential instead of relying on intuition-based development. Compare RAG approaches against long-context LLM alternatives using real benchmarks from Llama 4 and Gemini 2.5, and implement hybrid search strategies combining BM25 with similarity search and metadata filtering. Build a complete self-hosted multimodal RAG stack using Pixtral, Milvus, and vLLM while addressing the critical inference challenges of latency, throughput, and batching. Master advanced optimization techniques including model parallelism through tensor parallelism, quantization strategies, and paged attention with KV cache management. Follow along with a live demonstration of the complete architecture and setup, and participate in Q&A covering chunking strategies, CAG (Context-Augmented Generation), and other advanced topics essential for senior engineers and architects working with large-scale AI systems.