High-Throughput, Low-Latency Embedding Pipelines for Real-World Applications

Learn to build production-ready embedding pipelines that deliver high throughput and low latency for real-world applications in this 17-minute conference talk. Discover practical patterns from companies successfully running embedding inference at scale, including where performance bottlenecks typically occur and proven architectural solutions to address them. Explore critical considerations for model selection, dimensionality optimization, and quantization techniques that impact system performance. Examine open-source tools that can enhance any embedding API and gain deployment strategies for compound AI systems where multiple models and tools must coordinate effectively. Master the skills to diagnose pipeline bottlenecks, design resilient embedding architectures, and deliver faster systems while controlling costs for RAG, search, agents, and recommendation applications.