Introduction to Derivative-Free and Zeroth Order Optimization I

Explore derivative-free optimization methods that solve optimization problems without relying on derivative information in this mathematical lecture. Learn about truly zeroth-order methods including the Nelder-Mead method, direct search methods, and pattern search methods, examining their convergence properties and computational complexity. Discover how these approaches tackle optimization challenges when gradient information is unavailable or unreliable, making them essential tools for problems involving noisy, discontinuous, or black-box objective functions. Gain foundational understanding of this specialized optimization field through detailed analysis of method mechanics and theoretical guarantees, preparing you for advanced topics in model-based derivative-free approaches covered in subsequent lectures.