Multi-Precision Optimization Algorithm: Decreasing Computational Cost and Controlling Computational Error

Explore a 56-minute seminar from the "Meet a GERAD researcher!" series focusing on multi-precision optimization algorithms for minimizing smooth, non-convex functions. Delve into the Quadratic Regularization (R2) algorithm, a gradient descent method with adaptive step size, and its extension into a Multi-Precision (MPR2) version. Learn how MPR2 dynamically adapts precision levels to reduce computational effort while maintaining convergence to a minimum. Discover the challenges of variable precision computing and how MPR2 addresses them. Examine the algorithm's performance through various problem examples, with particular emphasis on applications like deep neural network training where computational cost reduction is crucial.