Approximation with Deep Networks - Remi Gribonval, Inria

Explore the intricacies of deep network approximation in this 50-minute conference talk by Remi Gribonval from Inria, presented at the Alan Turing Institute. Delve into the mathematics of data science, bridging computational statistics, machine learning, optimization, information theory, and learning theory. Begin with an introduction to feedforward neural networks and study the expressivity of deep neural networks (DNNs). Examine the ReLU activation function and its universal approximation properties. Investigate the benefits of sparsely connected networks and various network shapes. Compare direct and inverse estimates in approximation with sparse networks. Understand the role of skip-connections, neuron count vs. connections, and activation functions. Analyze spline activation functions and guidelines for choosing appropriate activations. Explore the benefits of network depth and its impact on approximation capabilities. Conclude with a comprehensive summary of approximation with DNNs and future perspectives in this cutting-edge field.