Generalization and Overfitting in Two-Layer Neural Networks

This lecture from the Simons Institute's Deep Learning Theory series features Pierfrancesco Urbani (CNRS) discussing the complex relationship between generalization and overfitting in two-layer neural networks. Explore how statistical physics techniques, particularly dynamical mean field theory, can be applied to study training dynamics in large, overparametrized neural networks. Understand key theoretical machine learning concepts including implicit bias hypothesis, benign overfitting, and feature learning regimes where neural networks identify latent data structures. The presentation details joint research with Andrea Montanari that provides insights into how generalization properties emerge in overparametrized models, addressing a central problem in theoretical machine learning.