Everything You Need to Know About Fine-Tuning and Merging LLMs

Learn the fundamentals of fine-tuning and merging large language models in this comprehensive conference talk that covers when fine-tuning is appropriate for customizing models to specific business needs. Explore the complete LLM training lifecycle and understand why enterprises prioritize open-source solutions for their AI implementations. Discover popular libraries for efficient fine-tuning and master key techniques including supervised fine-tuning methods like LoRA and QLoRA, as well as preference alignment approaches such as PPO, DPO, and KTO. Gain practical insights into creating high-quality supervised fine-tuning datasets, optimizing hyperparameters, and managing sequence length considerations. Delve into advanced model merging techniques including slurp, passthrough, and mixture of experts approaches that enable combining multiple specialized models effectively. Understand the strategic considerations that drive enterprise adoption of fine-tuning workflows and learn how to implement these techniques using modern tooling and frameworks.