Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

Stochastic Approximation: Theory and Applications

NPTEL via Swayam

Go to class Write review

ABOUT THE COURSE:Stochastic approximation refers to a class of iterative algorithms that can locate zeroes or optimal points of functions in scenarios where the function evaluations are compromised by noise. These algorithms are prominently utilized in regression, system identification, adaptive control, and increasingly in reinforcement learning and machine learning. This course will explore the design, theoretical convergence, and convergence rates of these algorithms, with a particular emphasis on applications in reinforcement learningINTENDED AUDIENCE: UG, Masters and Ph.D. students in Computer Science and Engineering/Electronics and Communication Engineerings/MathematicsPREREQUISITES: Real analysis, Measure-theoretic Probability, Optimization, Design and Analysis of Algorithms, Ordinary Differential EquationsINDUSTRY SUPPORT: Google Research, Microsoft Research, Adobe Research

Syllabus

Week 1:Introduction to stochastic approximation: Motivating examples from web-crawling and reinforcement learning
Week 2:Conditional expectation: Examples, Definition, Least-squares-best predictor, Existence, Properties
Week 3:Martingales: Filtration, Adapted Process, Definition, Examples, Convergence
Week 4:Ordinary differential equations: Existence and Uniqueness of solutions, Gronwall inequality, Asymptotic Behaviors, Invariant sets, Internally chain transitive sets
Week 5:Convergence of stochastic approximation algorithms: ODE method
Week 6:Convergence of stochastic approximation algorithms: ODE method (continue.)
Week 7:Convergence rates of linear stochastic approximation algorithms
Week 8:Stability of stochastic approximation algorithms
Week 9:Two-timescale stochastic approximation: Convergence
Week 10:Stochastic Recursive Inclusions: Convergence
Week 11:Applications to reinforcement learning
Week 12:Applications to reinforcement learning(continue.)