Challenges of Federated and Zero-Cost Neural Architecture Search

Explore a technical seminar that delves into two significant research areas in Neural Architecture Search (NAS): federated learning across diverse devices and zero-cost searching algorithms. Learn about innovative approaches to training diverse models across devices with varying computational capabilities using weight sharing techniques, demonstrating the possibility of collaborative training even with limited client resources. Discover insights into zero-cost proxies' effectiveness in NAS through quantitative and qualitative analysis, and understand a novel searching algorithm that combines zero-cost proxies with perturbation methods to efficiently identify high-performing subnets within a supernet. Presented by researcher Łukasz Dudziak, this hour-long talk provides valuable insights into addressing key challenges in modern machine learning architecture development.