Open Source LLMs: Costly Myths and Real-World Scaling

This conference talk reveals critical insights about deploying and scaling open-source Large Language Models (LLMs) for production workloads. Andrey Cheptsov, founder of dstack, examines the truth behind performance benchmarks, hardware requirements, and optimization techniques like quantization and LoRA. Discover when and why open-source LLMs can provide a competitive edge over proprietary models. Learn about the current state of open-source LLM performance compared to closed-source models, practical hardware considerations and optimization strategies for scaling, key frameworks and tools for pre-training, post-training, and inference, and strategic criteria for choosing open-source LLMs for business applications. The 44-minute presentation includes a transcript and invites viewers to share their own LLM optimization experiences.