Gradient Descent on Infinitely Wide Neural Networks - Global Convergence and

Explore gradient descent on infinitely wide neural networks in this 45-minute lecture by Francis Bach at the International Mathematical Union. Delve into the mathematical guarantees for convex optimization problems in linear parameter prediction models and the challenges of non-convex optimization in neural networks. Examine two-layer neural networks with homogeneous activation functions as the number of hidden neurons approaches infinity, and discover how qualitative convergence guarantees can be derived. Learn about machine learning contexts, parametric supervised learning, theoretical deep learning analysis, and optimization techniques for multi-layer neural networks. Investigate Wasserstein gradient flow, global convergence, and the transition from optimization to statistics. Compare kernel and feature learning regimes, and gain insights into the interpolation regime and logistic regression for two-layer neural networks.