Explore the theoretical foundations and practical applications of deep learning through this comprehensive workshop series from the Institute for Pure & Applied Mathematics at UCLA. Delve into the emergent properties of modern neural networks that operate at unprecedented scale, examining their success across natural language processing, structural biology, and computer vision. Investigate how these vast artificial intelligence systems challenge traditional machine learning beliefs about overfitting and high-dimensional optimization while giving rise to new empirical findings around feature learning, transfer learning, adversarial examples, scaling laws, and adaptive optimization methods. Examine the intricate relationships between model architecture, data, and optimizers at large scale through presentations from leading researchers in both theoretical and experimental domains. Learn about neural network feature learning mechanisms, infinite-width theory applications, large language model training methodologies, scaling laws mathematics, generative AI through dense associative memory, memorization and generalization dynamics, emergence and grokking phenomena, interpretability theory and applications, geometric representations in deep networks, mechanistic interpretation of neural computation, compositional structure exploitation, PAC-Bayes excess risk bounds, task comparison and transfer in overparameterized learning, brain network dynamics, implicit bias in robust machine learning, adaptive training method design, singular learning theory, and cooperative communication principles. Gain insights into the similarities and differences between biological and artificial learning systems while exploring how to frame fundamental questions about learning in the age of large-scale neural networks.

Syllabus

Adityanarayanan Radhakrishnan - How do neural networks learn features from data? - IPAM at UCLA
Leena Vankadara - Scaling Insights from Infinite-Width Theory for Next Gen Architecture & Learning
Sam Smith - How to train an LLM - IPAM at UCLA
Elvis Dohmatob - The Mathematics of Scaling Laws and Model Collapse in AI - IPAM at UCLA
Dmitry Krotov - Generative AI models through the lens of Dense Associative Memory - IPAM at UCLA
Blake Bordelon - Infinite limits and scaling laws of neural networks - IPAM at UCLA
Gintare Karolina Dziugaite - The dynamics of memorization and generalization in deep learning
Misha Belkin - Emergence and grokking in "simple" architectures - IPAM at UCLA
Oliver Eberle - Interpretability for Deep Learning: Theory, Applications and Scientific Insights
Boris Hanin - Neural Network Scaling Limits - IPAM at UCLA
Paul Riechers - geometric representation of far future in deep neural networks trained on next-token
Fanny Yang - Surprising phenomena of max-lp-margin classifiers in high dimensions - IPAM at UCLA
Cengiz Pehlevan - 2 stories in mechanistic interpretation of natural & artificial neural computation
Mauro Maggioni - On exploiting compositional structure: one bit of theory and one application
Dan Roy - Size of Teachers as Measure of Data Complexity: PAC-Bayes Excess Risk Bounds & Scaling Law
Vidya Muthukumar - Comparison and transfer between tasks in overparameterized learning
Mayank Mehta - Dynamics of brain's deep network - IPAM at UCLA
Nikos Tsilivis - The Price of Implicit Bias in Robust ML - IPAM at UCLA
Wu Lin - A framework for designing (non-diagonal) adaptive training methods - IPAM at UCLA
Shaowei Lin - Singular Learning, Relative Information and the Dual Numbers - IPAM at UCLA
Patrick Shafto - Common Ground in Cooperative Communication - IPAM at UCLA