Simons Institute Courses

Transformer Expressivity and Formal Logic

Explore the relationship between transformers and formal logic, uncovering their abilities, limitations, and implications for computational models.

• YouTube
• 46 minutes
• On-Demand
• Free Video

Using Recurrence to Achieve Weak to Strong Generalization

Explore recurrent architectures for weak-to-strong generalization in reasoning models, enhancing problem-solving capabilities beyond training data.

• YouTube
• 47 minutes
• On-Demand
• Free Video

Language Acquisition in Language Models

Explore language acquisition in masked language models, focusing on Syntactic Attention Structure and its impact on model development and linguistic capabilities.

• YouTube
• 45 minutes
• On-Demand
• Free Video

Do Large Language Models Perform Latent Reasoning?

Explore the potential for latent reasoning in large language models and their computational implications in this insightful remote talk by Mor Geva.

• YouTube
• 32 minutes
• On-Demand
• Free Video

Transformers as a Computational Model - What Was Revolutionized by the "Transformer Revolution?"

Explore the paradigm shift in machine learning brought by transformers, examining new research questions and recent findings in AI development.

• YouTube
• 44 minutes
• On-Demand
• Free Video

Iterated Models: Expressive Power, Learning, and Chain of Thought

Explore sequence-to-sequence models, focusing on transformers, their computational power, and learning complexities. Gain insights into iterated functions and chain of thought.

• YouTube
• 54 minutes
• On-Demand
• Free Video

A Retrieval-based Language Model at Scale - Remote Talk

Explore retrieval-based language models, their advantages over dense models, and recent improvements in pre-training and scaling for enhanced performance across various tasks.

• YouTube
• 39 minutes
• On-Demand
• Free Video

Using Algorithms to Understand Transformers and Using Transformers to Understand Algorithms

Explore algorithmic tools and optimization theory to uncover Transformer mechanisms for computational tasks, and discover how Transformers design data structures for tasks like nearest neighbor search.

• YouTube
• 48 minutes
• On-Demand
• Free Video

Towards Understanding Modern Alchemy - Transformers as a Computational Model

Explore in-context learning of formal languages, comparing Transformers to other models and examining specialized "n-gram heads" for improved performance in language modeling.

• YouTube
• 37 minutes
• On-Demand
• Free Video

Associative Memories as a Building Block in Transformers

Explore associative memories in Transformers, their role in storing knowledge, and how gradients and over-parameterization affect their learning and capacity.

• YouTube
• 39 minutes
• On-Demand
• Free Video

Towards Practical Distribution Testing

Explore practical algorithms for testing statistical correctness of samplers in high-dimensional settings, focusing on grey-box models and conditional sampling for efficient TV distance estimation.

• YouTube
• 23 minutes
• On-Demand
• Free Video

Tolerant Property Testing and Distance Approximation in Sublinear Algorithms

Explore tolerant property testing and distance approximation algorithms for various object types and properties. Gain insights into bipartiteness, monotonicity, uniformity, and subsequence-freeness.

• YouTube
• 59 minutes
• On-Demand
• Free Video

A Strong Separation for Adversarially Robust L0 Estimation for Linear Sketches

Explore adaptive attacks on linear sketches for L_0-estimation in turnstile streams, examining a novel approach that breaks sketches with high probability using ~O(r^8) queries.

• YouTube
• 30 minutes
• On-Demand
• Free Video

Toward Optimal Semi-streaming Algorithm for (1+ε)-Approximate Maximum Matching

Explore a deterministic algorithm for (1+ε)-approximate maximum matching, improving pass-complexity and advancing solutions in various computational models.

• YouTube
• 31 minutes
• On-Demand
• Free Video

Low Degree Testing over the Reals

Explore low degree testing over real numbers, focusing on theoretical concepts and applications in local algorithms.

• YouTube
• 33 minutes
• On-Demand
• Free Video