Efficient AI Serving at Scale - Processing Near Memory Acceleration for LLMs and Vector Search

Learn about cutting-edge processing near memory acceleration techniques for large language models and vector search applications in this 18-minute conference talk from the Open Compute Project. Discover how to achieve efficient AI serving at scale through innovative hardware acceleration approaches presented by senior engineering leaders from Marvell, including insights into memory-centric computing architectures that optimize performance for modern AI workloads. Explore the technical challenges and solutions for deploying LLMs and vector search systems at enterprise scale, with detailed discussions on how near-memory processing can dramatically improve throughput and reduce latency for AI inference tasks.