Coursera Flash Sale
40% Off Coursera Plus for 3 Months!
Grab it
Explore the dynamic privacy implications of personally identifiable information (PII) in large language model training through this 42-minute Google TechTalk presented by Jaydeep Borkar. Discover how PII memorization evolves throughout training pipelines and depends on commonly altered design choices, challenging the assumption that privacy risks are static. Learn about three novel phenomena that create unexpected privacy vulnerabilities: assisted memorization, where similar-appearing PII seen later in training can trigger recall of earlier sequences (accounting for up to one-third of memorization cases); amplified memorization, where adding new PII can increase memorization of existing PII by as much as 7.5 times; and compensatory memorization, where removing PII paradoxically leads to increased memorization of other personal information. Understand how these first- and second-order privacy risks emerge from routine training decisions including dataset curation changes, new data scraping for retraining, and downstream fine-tuning stages. Gain insights into the complex interplay between training dynamics and privacy preservation, essential for model creators seeking to minimize PII regurgitation risks while maintaining model performance.
