How Much Do Language Models Memorize?

Explore the fundamental question of language model memorization through a comprehensive technical presentation that introduces a novel method for measuring how much information models retain about specific datapoints. Learn to distinguish between unintended memorization (information about specific datasets) and generalization (information about true data-generation processes) through formal mathematical frameworks. Discover groundbreaking research findings showing that GPT-style models have an estimated capacity of approximately 3.6 bits per parameter when generalization is completely eliminated. Examine the relationship between model capacity, data size, and memorization through experiments involving hundreds of transformer language models ranging from small to large parameter counts. Understand the phenomenon of "grokking" - the point at which models transition from memorization to generalization as their capacity fills. Gain insights into scaling laws that relate model capacity and data size to membership inference, providing crucial understanding for privacy considerations in machine learning. Delve into the methodology for separating memorization components and measuring total model capacity, offering practical tools for evaluating modern language models' information retention capabilities.