What Are the Statistical Limits of Offline Reinforcement Learning With Function Approximation?

Explore the statistical boundaries of offline reinforcement learning with function approximation in this 55-minute lecture by Sham Kakade from the University of Washington and Microsoft Research. Delve into key concepts including realizability, sequential decision making, coverage limits, and policy evaluation. Examine upper and lower bounds, practical considerations, and experimental results. Gain insights into the mathematics of online decision making and the interplay between models and features in reinforcement learning.