Heterogeneity-Aware Algorithms for Federated Learning and Distributed Optimization

Learn about federated learning and distributed optimization in this technical lecture from Carnegie Mellon University Associate Professor Gauri Joshi. Explore the challenges and solutions for implementing machine learning at the edge, where data collection and model training occur on resource-constrained mobile devices. Dive into the complexities of heterogeneity in federated learning systems, examining how variations in data, communication, and computation across edge clients impact system performance. Discover recent algorithmic developments designed to address these heterogeneity challenges, including approaches to client selection and local adaptive optimization. Follow along as Prof. Joshi, an MIT Technology Review 35 Innovators under 35 recipient and NSF CAREER Award winner, breaks down key concepts from stochastic gradient descent to next word prediction applications, while addressing critical questions about scalability and flexibility in federated optimization systems.