Why Language Models Hallucinate

Explore the phenomenon of hallucination in large language models through this computer science seminar presented by Adam Kalai from OpenAI at the Institute for Advanced Study. Examine why LLMs generate plausible but factually incorrect statements, moving beyond viewing these as mysterious architectural failures to understanding them as predictable consequences of standard training practices. Learn how hallucinations can be conceptualized as classification errors, where models produce false statements when they cannot reliably distinguish truth from falsehood rather than expressing uncertainty. Discover how current benchmark optimization encourages guessing over abstention, as evaluation metrics typically penalize uncertainty expression. Investigate potential mitigation strategies through benchmark revision that rewards calibrated abstention, thereby realigning incentives in model development. This research collaboration with Santosh Vempala from Georgia Tech and Ofir Nachum & Edwin Zhang from OpenAI provides insights into improving the reliability and trustworthiness of language model outputs.