How Fast Are LLM Inference Engines Anyway?

Explore comprehensive benchmarking results comparing the performance of various LLM inference engines in this 16-minute conference talk. Discover how the landscape of open weights models and open source inference servers has dramatically evolved, creating an abundance of choices for AI engineers looking to self-host inference solutions. Learn from hundreds of benchmark runs conducted across different models, frameworks, and hardware configurations to understand which options deliver the best performance for your specific needs. Gain practical insights and proven tips from teams successfully deploying LLM inference at scale, helping you navigate the complex decision-making process when selecting the right inference engine for your applications.