Transformers Courses and Certifications

Simons Institute

Re-thinking Transformers: Searching for Efficient Linear Layers over a Continuous Space

Explore efficient alternatives to dense linear layers in neural networks, examining structured matrices and their compute-optimal scaling laws for improved performance in large language models.

• YouTube
• 42 minutes
• On-Demand
• Free Video

Center for Language & Speech Processing(CLSP), JHU

The NLP Task Effectiveness of Long-Range Transformers

Explore the effectiveness of long-range Transformer variants on NLP tasks, comparing their performance and analyzing attention behaviors to uncover strengths and limitations.

• YouTube
• 12 minutes
• On-Demand
• Free Video

Unify

Knowledge Circuits in Pretrained Transformers Explained

Explore knowledge circuits in pretrained transformers, uncovering computational mechanisms behind language models' articulation of specific knowledge. Gain insights into AI's inner workings.

• YouTube
• 1 hour
• On-Demand
• Free Video

Center for Language & Speech Processing(CLSP), JHU

Toward Length Extrapolatable Transformers

Explore innovative techniques for enhancing transformer models' ability to handle longer sequences, improving their performance and scalability in natural language processing tasks.

• YouTube
• 1 hour 5 minutes
• On-Demand
• Free Video

Simons Institute

Learning Theory of Transformers: Generalization and Optimization of In-Context Learning

Delve into theoretical insights on Transformer models' in-context learning capabilities, focusing on statistical efficiency, approximation abilities, and optimization guarantees in machine learning.

• YouTube
• 46 minutes
• On-Demand
• Free Video

Center for Language & Speech Processing(CLSP), JHU

Beyond Transformers - New Sequence Processing Architectures - Day 6 Morning

Explore cutting-edge sequence processing architectures that go beyond transformers, featuring insights from industry and academic experts on next-generation neural network designs.

• YouTube
• 3 hours 32 minutes
• On-Demand
• Free Video

Center for Language & Speech Processing(CLSP), JHU

Beyond Transformers - New Sequence Processing Architectures - Day 6 Morning

Explore cutting-edge sequence processing architectures that go beyond transformers, featuring evolutionary algorithms for automated design of neural networks and AI systems.

• YouTube
• 3 hours 33 minutes
• On-Demand
• Free Video

Simons Institute

How Do Transformers Learn Variable Binding? - A Study in Neural Symbolic Computation

Delve into the mechanisms behind variable binding in Transformer networks, exploring how neural architectures implement symbolic computation and track complex variable assignments.

• YouTube
• 1 hour 11 minutes
• On-Demand
• Free Video

Google TechTalks

Theoretical Limitations of Multi-Layer Transformers

Delve into groundbreaking research on multi-layer transformer limitations, exploring mathematical proofs for decoder-only architectures and their implications for language model development.

• YouTube
• 38 minutes
• On-Demand
• Free Video

AI Bites

Transformers without Normalization - Paper Explained

Explore how Transformers can function without normalization layers, challenging fundamental assumptions about Layer Norm and RMSNorm in modern LLMs like GPT-4 and LLAMA.

• YouTube
• 13 minutes
• On-Demand
• Free Video

UofU Data Science

Transformers Part 1

Discover the foundational concepts of transformer architecture and its revolutionary impact on natural language processing and machine learning applications.

• YouTube
• 1 hour 20 minutes
• On-Demand
• Free Video

UofU Data Science

Transformers Part 2

Dive into advanced transformer architectures and implementation techniques in this comprehensive deep learning session.

• YouTube
• 1 hour 19 minutes
• On-Demand
• Free Video

Valence Labs

Transformers Discover Molecular Structure Without Graph Priors

Discover how Transformers can learn molecular structure from Cartesian coordinates without graph priors, challenging GNN dominance in molecular machine learning and drug discovery.

• YouTube
• 1 hour 4 minutes
• On-Demand
• Free Video

Harvard CMSA

Can Transformers Reason Logically? A Study in SAT-Solving

Explore how Transformer models perform Boolean reasoning and solve 3-SAT problems, examining their capabilities, limitations, and potential solutions through Chain-of-Thought methodology and DPLL algorithms.

• YouTube
• 1 hour 3 minutes
• On-Demand
• Free Video

Harvard CMSA

How Far Can Transformers Reason? The Globality Barrier and Inductive Scratchpad

Delve into transformer models' reasoning capabilities, exploring globality barriers, syllogistic composition, and how inductive scratchpads can enhance learning and generalization in AI systems.

• YouTube
• 52 minutes
• On-Demand
• Free Video