OpenThoughts - Data Recipes for Reasoning Models

Explore the development of OpenThinker, a state-of-the-art small reasoning model that outperforms DeepSeek distill, through this 20-minute conference talk from the AI Engineer World's Fair. Discover the dataset recipe and post-training strategies used to build the strongest reasoning models, which you can adapt for your own domain-specific applications. Learn about the rigorous experimentation journey from thousands of data points with Bespoke-Stratos to millions of data points with OpenThinker3, including which scaling strategies work and which don't. Gain insights into open-source engineering solutions for large-scale synthetic data generation, training across multiple supercomputing clusters, and implementing fast, reliable evaluations. The presentation covers the problem of open-source reasoning in AI models, the effectiveness of Supervised Fine-Tuning (SFT) for reasoning tasks, OpenThoughts 3's performance metrics, key learnings from data recipe development, guidance for adapting the dataset recipe to specific domains, and opportunities for open collaboration. Presented by Ryan Marten, co-lead of the OpenThinker collaboration and founding engineer at Bespoke Labs, who brings extensive AI research experience from institutions including University of Illinois Urbana-Champaign, University of Toronto, University of Oxford, AI2, and Vector Institute.