From Large Language Models to Reasoning Language Models - Three Eras in the Age of Computation

Explore a comprehensive technical talk that traces the evolutionary journey of Large Language Models through computational and optimization perspectives. Learn about the foundational developments in LLMs, examining how computational and optimization advances played crucial roles in their creation. Discover the optimization techniques that achieved 1000x cost reduction, making these models accessible on mobile devices. Understand the concept of constructive hallucination as a solution to human-generated data limitations, enabling new hypothesis generation and validation through reasoning chains. Examine the technological foundations and early achievements of reasoning models like OpenAI's o1 and o3 preview, while considering their increased computational requirements. Get insights into the Ultra Ethernet initiative, designed to establish interconnect standards for future AI workloads, addressing system-level demands in the reasoning model era.