Positional Encoding Courses and Certifications

Yannic Kilcher

Train Short, Test Long - Attention With Linear Biases Enables Input Length Extrapolation

ALiBi: A novel attention mechanism enabling transformers to process longer sequences than trained on. Uses linear biases instead of position encodings, improving efficiency and extrapolation capabilities.

• YouTube
• 31 minutes
• Self-Paced
• Free Video

Yannic Kilcher

Perceiver - General Perception with Iterative Attention

Explore DeepMind's Perceiver model, a versatile architecture that processes diverse input modalities using iterative attention, solving the quadratic bottleneck problem in Transformers.

• YouTube
• 30 minutes
• Self-Paced
• Free Video

Yannic Kilcher

DeBERTa - Decoding-Enhanced BERT with Disentangled Attention

Explore DeBERTa, an advanced BERT model with disentangled attention and enhanced mask decoder, surpassing RoBERTa in NLP tasks. Learn about its architecture, improvements, and performance gains.

• YouTube
• 45 minutes
• Self-Paced
• Free Video

Yannic Kilcher

LambdaNetworks- Modeling Long-Range Interactions Without Attention

Explore LambdaNetworks, a novel approach to modeling long-range interactions in AI without traditional attention mechanisms, offering improved efficiency and performance in computer vision tasks.

• YouTube
• 1 hour
• Self-Paced
• Free Video

Aleksa Gordić - The AI Epiphany

Attention Is All You Need - Transformer Paper Explained

Detailed walkthrough of the original transformer architecture, covering tokenization, embeddings, attention mechanisms, and decoding, with a focus on machine translation applications.

• YouTube
• 39 minutes
• Self-Paced
• Free Video

StatQuest with Josh Starmer

Transformer Neural Networks, ChatGPT's Foundation, Clearly Explained

Clear explanation of Transformer Neural Networks, the foundation of ChatGPT and modern AI. Covers key concepts like word embedding, self-attention, and encoder-decoder architecture.

• YouTube
• 36 minutes
• Self-Paced
• Free Video

Outlier

RoPE - Rotary Positional Embeddings Explanation and PyTorch Implementation

Discover how RoPE (Rotary Positional Embeddings) teaches Transformers about data positions in text, images, and videos with clear explanations and PyTorch code implementation.

• YouTube
• 20 minutes
• Self-Paced
• Free Video

The Full Stack

LLM Foundations - LLM Bootcamp

Explore foundational concepts of large language models, including machine learning basics, Transformer architecture, and notable LLMs. Gain insights into pretraining datasets and instruction tuning.

• YouTube
• 48 minutes
• Self-Paced
• Free Video

The Full Stack

Transfer Learning and Transformers - Full Stack Deep Learning - Spring 2021

Explore transfer learning's evolution from computer vision to NLP, diving into embeddings, language models, and the revolutionary Transformer architecture. Gain insights into various models and their applications.

• YouTube
• 49 minutes
• Self-Paced
• Free Video

CodeEmporium

Transformers - From Zero to Hero

Comprehensive deep dive into transformer architecture, covering key concepts from self-attention to tokenization. Includes hands-on coding for both encoder and decoder implementations.

• YouTube
• 3 hours 35 minutes
• Self-Paced
• Free Video

StatQuest with Josh Starmer

Decoder-Only Transformers, ChatGPT's Specific Transformer, Clearly Explained

Comprehensive explanation of Decoder-Only Transformers used in ChatGPT, covering word embedding, position encoding, masked self-attention, and output generation, with comparisons to normal Transformers.

• YouTube
• 37 minutes
• Self-Paced
• Free Video

StatQuest with Josh Starmer

Coding a ChatGPT-Like Transformer From Scratch in PyTorch

Step-by-step tutorial on building a ChatGPT-like Transformer from scratch in PyTorch, covering data preparation, position encoding, attention mechanisms, and model training for natural language processing.

• YouTube
• 31 minutes
• Self-Paced
• Free Video

Serrano.Academy

Transformer Models: Understanding Their Architecture and Functionality - Part 3

Demystifying Transformer models with visuals and examples, covering key concepts like attention mechanism, tokenization, embeddings, and fine-tuning for various NLP tasks.

• YouTube
• 44 minutes
• Self-Paced
• Free Video

MIT HAN Lab

Vision Transformer - Lecture 14

Explore Vision Transformers in deep learning, covering architecture, training, and applications. Gain insights into this powerful model for computer vision tasks.

• YouTube
• 1 hour 14 minutes
• Self-Paced
• Free Video

MIT HAN Lab

Transformer and Large Language Models - Part I - Lecture 12

Explore Transformer architecture and Large Language Models in this comprehensive MIT lecture, covering key concepts and recent advancements in AI.

• YouTube
• 1 hour 18 minutes
• Self-Paced
• Free Video