Compositionality and Phase Transitions in Language Models - Understanding Knowledge Acquisition

Watch a 57-minute research lecture from the Simons Institute featuring Naomi Saphra from the Kempner Institute at Harvard University exploring the relationship between compositionality and phase transitions in language models. Delve into the intersection of large language models, cognitive science, linguistics, and neuroscience as Saphra examines how these models process and combine information. Gain insights into whether language models exhibit true compositional learning and understanding, or if their capabilities emerge through different mechanisms. The talk bridges theoretical concepts with practical implications for understanding how artificial neural networks process language compared to human cognition.