Google, IBM & Meta Certificates — All 10,000+ Courses at 40% Off
One annual plan covers every course and certificate on Coursera. 40% off for a limited time.
Get Full Access
Explore a detailed research seminar where CMU Machine Learning Professor and Microsoft Research scientist Yuanzhi Li delves into the complexities of knowledge management in Large Language Models (LLMs). Learn about groundbreaking experiments using synthetic biography datasets that reveal surprising limitations in how LLMs store and retrieve information, even when achieving perfect training performance. Discover the critical role of data augmentation techniques in improving knowledge storage within token embeddings, and understand why LLMs struggle with knowledge manipulation without chain-of-thought reasoning. Examine the relationship between model size and knowledge capacity, investigating whether current parameter scales are sufficient for storing human-level knowledge. Gain insights into fundamental questions about LLM architecture, training methodologies, and the physics-inspired approach to understanding these powerful AI systems.
