Fifty Shades of Caching and How LLMs Paint It Black

Explore the intricate world of caching technologies in this conference talk that examines how industry-standard caching systems are being challenged by the surge in Large Language Model (LLM) traffic. Delve into the fundamental role of caching as the web's quiet workhorse, from edge CDNs to per-process memory stores, and understand how these systems accelerate content delivery while maintaining website resilience under heavy traffic loads. Discover the practical implications of various caching technologies through real-world, open-source examples structured around comprehensive chapters. Learn about the current challenges facing caching infrastructure as LLM traffic patterns increasingly disrupt traditional caching strategies, and gain insights into how these machine learning workloads are fundamentally changing the caching landscape. Examine both the underlying technology principles and the evolving challenges that site reliability engineers face in maintaining optimal performance in an AI-driven web environment.