Building Reliable Machine Learning Systems - Challenges and Approaches

Explore a comprehensive lecture on building reliable machine learning systems in this Allen School Colloquium featuring Pang Wei Koh from Stanford University. Delve into the challenges of unreliable ML systems and their potential for catastrophic failures on specific data subpopulations or in different deployment environments. Learn about innovative approaches to enhance ML reliability, including the use of influence functions to understand model predictions and failures, and the application of distributionally robust optimization for improved performance across subpopulations. Discover the WILDS benchmark, which tests model robustness against real-world distribution shifts in various fields such as pathology, conservation, remote sensing, and drug discovery. Gain insights into the development of more dependable COVID-19 models using anonymized cellphone mobility data to inform public health policy, addressing challenges like changing environments and demographic heterogeneity. This 59-minute talk provides valuable knowledge for researchers and practitioners seeking to create more robust and reliable machine learning systems for real-world applications.