Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

NPTEL

Stochastic Approximation: Theory and Applications

NPTEL via Swayam

Go to class Write review
Coursera Ad

Learn AI, Data Science & Business — Earn Certificates That Get You Hired

Learn More → OpenLearning Ad

Future-Proof Your Career: AI Manager Masterclass

Learn More →

Overview

Google, IBM & Meta Certificates – 40% Off
One Coursera Plus subscription covers most Professional Certificates on Coursera.
Unlock All Certificates
ABOUT THE COURSE:Stochastic approximation refers to a class of iterative algorithms that can locate zeroes or optimal points of functions in scenarios where the function evaluations are compromised by noise. These algorithms are prominently utilized in regression, system identification, adaptive control, and increasingly in reinforcement learning and machine learning. This course will explore the design, theoretical convergence, and convergence rates of these algorithms, with a particular emphasis on applications in reinforcement learningINTENDED AUDIENCE: UG, Masters and Ph.D. students in Computer Science and Engineering/Electronics and Communication Engineerings/MathematicsPREREQUISITES: Real analysis, Measure-theoretic Probability, Optimization, Design and Analysis of Algorithms, Ordinary Differential EquationsINDUSTRY SUPPORT: Google Research, Microsoft Research, Adobe Research

Syllabus

Week 1:Introduction to stochastic approximation: Motivating examples from web-crawling and reinforcement learning
Week 2:Conditional expectation: Examples, Definition, Least-squares-best predictor, Existence, Properties
Week 3:Martingales: Filtration, Adapted Process, Definition, Examples, Convergence
Week 4:Ordinary differential equations: Existence and Uniqueness of solutions, Gronwall inequality, Asymptotic Behaviors, Invariant sets, Internally chain transitive sets
Week 5:Convergence of stochastic approximation algorithms: ODE method
Week 6:Convergence of stochastic approximation algorithms: ODE method (continue.)
Week 7:Convergence rates of linear stochastic approximation algorithms
Week 8:Stability of stochastic approximation algorithms
Week 9:Two-timescale stochastic approximation: Convergence
Week 10:Stochastic Recursive Inclusions: Convergence
Week 11:Applications to reinforcement learning
Week 12:Applications to reinforcement learning(continue.)

Taught by

Prof. Gugan Chandrashekhar Mallika Thoppe

Reviews

Start your review of Stochastic Approximation: Theory and Applications

Go to class

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Sign up for free
Someone learning on their laptop while sitting on the floor.