Introduction to Derivative-Free and Zeroth Order Optimization II

Explore advanced model-based approaches in derivative-free optimization through this lecture that builds upon foundational zeroth-order methods. Delve into sophisticated techniques that construct models of objective functions to approximate first and second derivatives, moving beyond purely zeroth-order approaches like Nelder-Mead, direct search, and pattern search methods. Learn how these model-building strategies enhance optimization performance when gradient information is unavailable or unreliable, examining their theoretical foundations, practical implementations, and convergence properties. Understand the trade-offs between computational efficiency and approximation accuracy in these derivative-free frameworks, and discover how model-based methods bridge the gap between classical optimization theory and real-world applications where derivative computation is prohibitively expensive or impossible. Gain insights into the mathematical foundations underlying these approaches and their applications across various domains where traditional gradient-based methods are not feasible.