Singular Learning Theory in Machine Learning - Part 2

Explore a mathematical seminar lecture where Dr. Simon Pepin lehalleur from the Universiteit van Amsterdam delves into Watanabe's findings in singular learning theory and their applications to machine learning. Learn how to estimate the local real log-canonical threshold of model parameters, a method that remains computationally feasible even for large-scale deep learning models with millions of parameters. Discover how this statistical complexity measure can be monitored during the learning process and understand its role in investigating the relationship between Watanabe's Bayesian framework and optimization algorithms like stochastic gradient descent. Examine the emergence of computational structure in phase transitions during neural network training through the lens of Developmental Interpretability, building upon the collaborative work of Murfet and team, which extends Watanabe's original research.