Large Scale Machine Learning and Convex Optimization - Lecture 3

Explore the intricacies of large-scale machine learning and convex optimization in this comprehensive lecture. Delve into the challenges of handling big data in machine learning and signal processing, focusing on online algorithms like stochastic gradient descent. Examine the optimal convergence rates for general convex and strongly-convex functions. Discover how the smoothness of loss functions can be leveraged to design innovative algorithms with improved performance. Learn about a novel Newton-based stochastic approximation algorithm that achieves faster convergence rates without strong convexity assumptions. Investigate the practical applications of combining batch and online algorithms for strongly convex problems. Cover topics such as subgradient descent, stochastic approximation, adaptive algorithms for logistic regression, self-concordance, and least-mean-square algorithms. Gain insights through theoretical proofs and synthetic example simulations.