Implicit Regularization Courses and Certifications

Simons Institute

Implicit Regularization II

Explore implicit regularization in deep learning, focusing on advanced concepts and their impact on model performance and generalization.

• YouTube
• 1 hour 23 minutes
• On-Demand
• Free Video

Simons Institute

Implicit Regularization I

Explore implicit regularization in deep learning, covering boosting, complexity control, optimization landscapes, and stochastic gradient descent for effective model training.

• YouTube
• 1 hour 17 minutes
• On-Demand
• Free Video

Alan Turing Institute

Implicit Regularization for General Norms and Errors - Lorenzo Rosasco, MIT

Explores implicit regularization in optimization methods, focusing on stability in estimation processes for non-Euclidean norms and non-quadratic errors, with emphasis on accelerated techniques.

• YouTube
• 41 minutes
• On-Demand
• Free Video

Hausdorff Center for Mathematics

Generalization in Deep Learning Through the Lens of Implicit Rank Minimization

Explores implicit regularization in deep learning through matrix and tensor factorizations, revealing tendencies towards low ranks. Discusses implications for generalization and potential improvements in neural network design.

• YouTube
• 41 minutes
• On-Demand
• Free Video

MITCBMM

Towards Understanding the Implicit Regularization Effect of SGD

Explore the implicit regularization effect of Stochastic Gradient Descent, uncovering its impact on optimization and generalization in machine learning models.

• YouTube
• 1 hour 14 minutes
• On-Demand
• Free Video

Institute for Pure & Applied Mathematics (IPAM)

Implicit and Explicit Regularization in Deep Neural Networks

Explores implicit and explicit regularization in deep neural networks, connecting learning algorithms to H-infinity control and explaining convergence behavior in over-parametrized models, offering insights into generalization abilities.

• YouTube
• 37 minutes
• On-Demand
• Free Video

Georgia Tech Research

Random Initialization and Implicit Regularization in Nonconvex Statistical Estimation - Lecture 2

Explore nonconvex optimization in statistical estimation, focusing on gradient descent's convergence to optimal solutions without explicit regularization or careful initialization.

• YouTube
• 48 minutes
• On-Demand
• Free Video

Fields Institute

Redundancy and Implicit Regularization in Neural Network Models

Explore how neural networks select specific parameter representations from equivalent options, examining fiber geometry in deep linear networks and ReLU architectures.

• YouTube
• 55 minutes
• On-Demand
• Free Video

Simons Institute

Why Deep Learning Works: Implicit Self-Regularization in Deep Neural Networks

Explore the underlying mechanisms of deep learning's success, focusing on implicit self-regularization in neural networks and its impact on model performance.

• YouTube
• 38 minutes
• On-Demand
• Free Video

Society for Industrial and Applied Mathematics

Optimal Transport for Inverse Problems and Implicit Regularization

Exploring optimal transport theory for inverse problems, addressing nonconvexity and noise sensitivity in least-squares methods, with applications in industry and broader data-fitting challenges.

• YouTube
• 1 hour 11 minutes
• On-Demand
• Free Video

Harvard CMSA

Implicit Regularization via Uniform Convergence in Linear Models and Kernel Methods

Explore the statistical analysis of mirror descent algorithms and their convergence properties in machine learning, focusing on early-stopped methods and uniform learning theory applications.

• YouTube
• 55 minutes
• On-Demand
• Free Video

Yannic Kilcher

Training More Effective Learned Optimizers, and Using Them to Train Themselves - Paper Explained

Explores a novel approach to machine learning optimization using neural networks, discussing its architecture, training process, and potential to revolutionize model training through self-improvement and generalization.

• YouTube
• 54 minutes
• On-Demand
• Free Video

Understanding Deep Learning Requires Rethinking Generalization

Explore deep learning generalization, examining regularization techniques, randomization tests, and neural network expressivity to challenge conventional understanding and improve model performance.

• YouTube
• 40 minutes
• On-Demand
• Free Video

Alan Turing Institute

Benign Overfitting - Peter Bartlett, UC Berkeley

Explore benign overfitting in machine learning, focusing on linear regression, deep networks, and statistical implications. Gain insights into effective rank, regularization, and future research directions.

• YouTube
• 42 minutes
• On-Demand
• Free Video

Alan Turing Institute

Big Data Is Low Rank - Madeleine Udell, Cornell University

Explore big data's low-rank structure, generalized low-rank models, and their applications in machine learning. Learn about dimensionality reduction, autoML, and latent variable models for efficient data analysis.

• YouTube
• 46 minutes
• On-Demand
• Free Video