Stochastic Gradient Descent Courses and Certifications

VinAI

Stochastic Gradient Descent Methods with Biased Estimators

Explore recent advancements in stochastic gradient descent methods, focusing on biased estimators for large-scale optimization and minimax problems in machine learning applications.

• YouTube
• 1 hour 7 minutes
• On-Demand
• Free Video

Erwin Schrödinger International Institute for Mathematics and Physics (ESI)

The Long-Run Distribution of Stochastic Gradient Descent: A Large Deviations Approach

Explore the long-run behavior of stochastic gradient descent in non-convex problems, analyzing state space visitation patterns and energy level distributions.

• YouTube
• 25 minutes
• On-Demand
• Free Video

Hausdorff Center for Mathematics

Multilevel Stochastic Gradient Descent for Risk-Averse PDE Constraint Optimization

Explore multilevel stochastic gradient descent for optimizing PDE-constrained systems with uncertain data, featuring risk-averse objectives and parallel Monte Carlo estimators.

• YouTube
• 43 minutes
• On-Demand
• Free Video

UofU Data Science

Stochastic Gradient Descent for SVM - Lecture 21

Explore stochastic sub-gradient descent as an optimization strategy for SVM loss functions and its connection to the perceptron algorithm.

• YouTube
• 1 hour 20 minutes
• On-Demand
• Free Video

Why Does Adam Work So Well for LLMs? And Can We Find Optimal Per-Variable Step Sizes

Explore why Adam optimizer outperforms SGD in large language models due to class imbalance rather than heavy-tailed noise, plus discover new methods for optimal step sizes.

• YouTube
• 1 hour 10 minutes
• On-Demand
• Free Video

Society for Industrial and Applied Mathematics

Bayesian Inference with Plug-and-Play Priors - Imaging and Inverse Problems Seminar

Explore Bayesian imaging with Plug & Play methods, focusing on convergence guarantees and well-posed models. Learn about novel algorithms for Monte Carlo sampling and MAP inference in inverse problems.

• YouTube
• 53 minutes
• On-Demand
• Free Video

Erwin Schrödinger International Institute for Mathematics and Physics (ESI)

The First Optimal Distributed SGD in the Presence of Data, Compute and Communication Heterogeneity

Explore optimal time complexities for parallel optimization methods with heterogeneous data, compute, and communication. Gain insights into efficient distributed SGD algorithms.

• YouTube
• 29 minutes
• On-Demand
• Free Video

Erwin Schrödinger International Institute for Mathematics and Physics (ESI)

Hessian-Aware Stochastic Differential Equation Modeling of SGD

Explore a novel SDE model for SGD that incorporates Hessian information, improving accuracy in capturing escaping behaviors and achieving exact recovery for quadratic objectives.

• YouTube
• 29 minutes
• On-Demand
• Free Video

INI Seminar Room 2

Stochastic Gradient with Least-Squares Control Variates

Explore advanced stochastic gradient optimization techniques using least-squares control variates to reduce variance and improve convergence in machine learning algorithms.

• YouTube
• 1 hour 3 minutes
• On-Demand
• Free Video

INI Seminar Room 2

Stochastic Gradient with Least-Squares Control Variates

Explore advanced stochastic gradient methods using least-squares control variates to reduce variance in optimization algorithms for machine learning and uncertainty quantification.

• YouTube
• 1 hour
• On-Demand
• Free Video

Federated Learning One World Seminar

The First Optimal Parallel SGD - Seminar #125

Discover the first optimal parallel SGD algorithm with Peter Richtarik from KAUST in this FLOW seminar, exploring groundbreaking advancements in federated learning optimization techniques.

• YouTube
• 1 hour 43 minutes
• On-Demand
• Free Video

Federated Learning One World Seminar

The First Asynchronous SGD with Optimal Time Complexity - Seminar #126

Discover the first asynchronous SGD algorithm with optimal time complexity as Arto Maranjyan presents groundbreaking research in federated learning at the FLOW Seminar #126.

• YouTube
• 59 minutes
• On-Demand
• Free Video

HUJI Machine Learning Club

SGD Exact Dynamics in High-Dimension - Insights for Algorithm Design

Explore SGD dynamics in high-dimensional regimes through theoretical frameworks, covering generalized linear models, adaptive methods, and differentially private optimization insights.

• YouTube
• 1 hour 2 minutes
• On-Demand
• Free Video

Alfredo Canziani

Stochastic Gradient Descent and Backpropagation

Explore stochastic gradient descent, backpropagation, and neural network implementation in PyTorch. Learn practical tricks, basic modules, and gradient computation for effective deep learning.

• YouTube
• 1 hour 43 minutes
• On-Demand
• Free Video

Great Learning

Stochastic Gradient Descent

Explore stochastic gradient descent, its advantages, and implementation. Learn about mini-batches, momentum, and convergence in this optimization algorithm for machine learning models.

• YouTube
• 1 hour 37 minutes
• On-Demand
• Free Video