Exploring the High-dimensional Random Landscapes of Data Science - 2/3

Explore the complex optimization challenges in machine learning and data science through this comprehensive lecture that examines high-dimensional random landscapes and their topological properties. Begin with an accessible introduction to the framework covering typical machine learning tasks and neural network structures before delving into the classical context of Stochastic Gradient Descent (SGD) in finite dimensions. Investigate the Tensor PCA model as a key example, understanding its relationship to spherical spin glasses and examining how simple algorithms perform in single spike estimation tasks. Progress to single index models and discover the concept of "effective dynamics" and "summary statistics" that operate in reduced dimensions to govern algorithm performance. Learn how systems identify these summary statistics through dynamical spectral transitions, where Gram matrices and Hessian matrices develop outliers along optimization trajectories. Master essential Random Matrix Theory tools including edge spectrum behavior and the BBP transition within broader mathematical contexts. Apply these concepts to practical machine learning examples including multilayer neural networks for Gaussian mixture classification, XOR problems, and multi-spike Tensor PCA models, drawing from cutting-edge research collaborations with leading experts in the field.