Simons Institute Courses

Towards Understanding Generalization Properties of Score-Based Losses

Explore score-based losses as alternatives to maximum likelihood in generative models, examining their statistical efficiency and connections to diffusion algorithms for improved model design.

• YouTube
• 1 hour 17 minutes
• On-Demand
• Free Video

Reconsidering Overfitting in the Age of Overparameterized Models

Explore modern machine learning paradigms, focusing on overparameterized models' ability to generalize well despite near-zero training error, challenging classical overfitting concepts.

• YouTube
• 1 hour 18 minutes
• On-Demand
• Free Video

Prediction, Generalization, and Complexity in Statistical Decision Theory - Part 2

Explore statistical decision theory's insights on prediction error, generalization gap, and model complexity. Examine fixed-X vs. random-X perspectives and their implications for machine learning.

• YouTube
• 1 hour 2 minutes
• On-Demand
• Free Video

Understanding Generalization from Pre-training Loss to Downstream Tasks

Explore the mystery of pre-trained models' effectiveness, examining contrastive loss, embedding spaces, and the roles of inductive bias and optimizers in generalization to downstream tasks.

• YouTube
• 1 hour 18 minutes
• On-Demand
• Free Video

A Theory of Multi-objective Machine Learning

Explore multi-objective learning as a unifying paradigm for robustness, collaboration, and fairness in machine learning, with technical tools and empirical evidence.

• YouTube
• 1 hour 11 minutes
• On-Demand
• Free Video

Learning Probability Distributions: What Can and Can't Be Done

Explore statistical learning, focusing on guaranteed generalizations from finite samples and characterizing learnable distribution families. Gain insights into modern paradigms in generalization.

• YouTube
• 1 hour 12 minutes
• On-Demand
• Free Video

The Elusive Generalization: Classical Bounds to Double Descent to Grokking

Explore the evolution of generalization in machine learning, from classical bounds to double descent and grokking, challenging traditional assumptions and practices in the field.

• YouTube
• 1 hour 20 minutes
• On-Demand
• Free Video

Statistical Limits of Causal Inference

Explore fundamental limits of causal effect estimation in observational data, examining personalized effects, non-smooth settings, and discrete scenarios without assuming prior causal inference knowledge.

• YouTube
• 1 hour 11 minutes
• On-Demand
• Free Video

Generalization in Natural Language Processing

Explore types of generalization in NLP, motivations, assumptions, and implications for language technology development. Gain insights into natural language's inherent generalization demands.

• YouTube
• 1 hour 16 minutes
• On-Demand
• Free Video

Modern Paradigms of Generalization - The Heliocentric Model of Aristarchus

Explore modern generalization paradigms and historical scientific models in this insightful lecture on machine learning theory and its historical parallels.

• YouTube
• 1 hour 10 minutes
• On-Demand
• Free Video

Towards Practical Distribution Testing

Explore practical algorithms for testing statistical correctness of samplers in high-dimensional settings, focusing on grey-box models and conditional sampling for efficient TV distance estimation.

• YouTube
• 23 minutes
• On-Demand
• Free Video

Tolerant Property Testing and Distance Approximation in Sublinear Algorithms

Explore tolerant property testing and distance approximation algorithms for various object types and properties. Gain insights into bipartiteness, monotonicity, uniformity, and subsequence-freeness.

• YouTube
• 59 minutes
• On-Demand
• Free Video

A Strong Separation for Adversarially Robust L0 Estimation for Linear Sketches

Explore adaptive attacks on linear sketches for L_0-estimation in turnstile streams, examining a novel approach that breaks sketches with high probability using ~O(r^8) queries.

• YouTube
• 30 minutes
• On-Demand
• Free Video

Toward Optimal Semi-streaming Algorithm for (1+ε)-Approximate Maximum Matching

Explore a deterministic algorithm for (1+ε)-approximate maximum matching, improving pass-complexity and advancing solutions in various computational models.

• YouTube
• 31 minutes
• On-Demand
• Free Video

Low Degree Testing over the Reals

Explore low degree testing over real numbers, focusing on theoretical concepts and applications in local algorithms.

• YouTube
• 33 minutes
• On-Demand
• Free Video