Access AI Models Anywhere - Scaling AI Traffic With Envoy AI Gateway

Learn how to deploy, scale, and manage access to diverse AI models across cloud and on-premises environments using Envoy AI Gateway in this conference talk from KubeCon + CloudNativeCon. Discover how organizations can address the accelerating challenges of Generative AI adoption through Envoy Proxy's powerful filter architecture and extensibility via ext-proc. Explore key features including centralized credential management, intelligent model routing, and LLM token usage control that make Envoy AI Gateway the first CNCF-backed open source AI gateway solution. Dive deep into the architecture that extends Envoy's capabilities to efficiently manage AI-driven workloads for enterprise needs while providing robustness, scalability, and adaptability in the rapidly-changing generative AI landscape. Watch a live demonstration of an AI agent seamlessly accessing models anywhere through a unified API, and understand how this solution democratizes AI infrastructure for organizations of all sizes by building on top of the high-performance Envoy Gateway foundation.