Weak to Strong Generalization in Random Feature Models

Join this Distinguished Seminar in Optimization & Data featuring Nathan Srebro from the Toyota Technological Institute at Chicago as he presents "Weak to Strong Generalization in Random Feature Models." Discover how the phenomenon of weak-to-strong generalization can occur in simple random feature models without requiring complex learners like GPT-4 or pre-training. Learn how a student model with more random features can outperform a teacher model despite being trained only on data labeled by that teacher. Srebro explains the crucial role of early stopping in enabling this phenomenon and presents the quantitative limits of weak-to-strong generalization. The talk includes joint work with Marko Medvedev, Kaifeng Lyu, Dingli Yu, Sanjeev Arora, and Zhiyuan Li. Nathan Srebro is a distinguished professor with significant contributions to machine learning, including work on Markov networks, nuclear norm applications, optimization techniques, fairness measures, and implicit optimization bias in deep learning, recognized through multiple Best Paper awards at major conferences.