Dynamo KVBM - Managing Memory at Scale

Learn about Dynamo's KV Block Manager (KVBM), a system designed to decouple memory management from run-time logic for memory-efficient, high-throughput LLM systems in this 29-minute technical session. Explore the architecture of KVBM and understand how it addresses KV cache management challenges in LLM deployments. Discover integration strategies with LLM inference runtimes and NIXL, while gaining insights into the development roadmap from the engineers building this system. Engage with practical implementation details and contribute to the open-source project through interactive discussion and Q&A with the development team.