Scientific Understanding of Neural Networks - From Representation to Learning Dynamics and From Shallow to Deep

Explore the scientific foundations of neural networks through this comprehensive conference talk that examines both representation theory and learning dynamics across shallow and deep architectures. Delve into the mathematical principles underlying neural network behavior, investigating how these systems represent information and evolve during the learning process. Analyze the fundamental differences between shallow and deep neural networks, understanding their respective capabilities and limitations from a theoretical perspective. Gain insights into the mathematical frameworks that govern neural network functionality, including representation capacity, optimization landscapes, and the dynamics of gradient-based learning algorithms. Examine how depth affects network expressivity and learning efficiency, while exploring the theoretical connections between network architecture and performance. This presentation provides a rigorous mathematical treatment of neural networks, bridging the gap between practical applications and theoretical understanding in machine learning.