Self-Attention Courses and Certifications

Yacine Mahdid

Masked Self-Attention from Scratch in Python

Dive into masked self-attention algorithm implementation in Python with numpy, covering theory and step-by-step coding from QKV computation to output generation, essential for understanding LLM pre-training.

• YouTube
• 14 minutes
• Self-Paced
• Free Video

Simons Institute

The Emergence of Clusters in Self-Attention Dynamics

Explore the formation of clusters in self-attention dynamics, examining their impact on transformer models and computational processes.

• YouTube
• 49 minutes
• Self-Paced
• Free Video

StatQuest with Josh Starmer

Transformer Neural Networks, ChatGPT's Foundation, Clearly Explained

Clear explanation of Transformer Neural Networks, the foundation of ChatGPT and modern AI. Covers key concepts like word embedding, self-attention, and encoder-decoder architecture.

• YouTube
• 36 minutes
• Self-Paced
• Free Video

Yannic Kilcher

Nyströmformer- A Nyström-Based Algorithm for Approximating Self-Attention

Explore Nyströmformer, a novel algorithm for approximating self-attention in Transformers with linear complexity, enabling processing of longer sequences and improving scalability in natural language tasks.

• YouTube
• 48 minutes
• Self-Paced
• Free Video

Donato Capitella

The Transformer Architecture: Understanding Self-Attention and Positional Encoding

Delve into the revolutionary Transformer architecture, exploring self-attention mechanisms, positional encoding, and key components that revolutionized natural language processing and machine learning.

• YouTube
• 15 minutes
• Self-Paced
• Free Video

Transformer Models with PyTorch

What makes LLMs tick? Discover how transformers revolutionized text modeling and kickstarted the generative AI boom.

• DataCamp
• 2 hours
• Self-Paced
• Free Trial Available

Neural Breakdown with AVB

Vision Transformers: Understanding Self-Attention and Implementation in PyTorch

Dive into Vision Transformers through clear visualizations and hands-on PyTorch implementation, mastering self-attention mechanisms and comparing them with CNNs in computer vision tasks.

• YouTube
• 16 minutes
• Self-Paced
• Free Video

Neural Breakdown with AVB

Understanding Self-Attention in Transformer Models - Part 2

Dive into the mechanics and power of Self-Attention in Transformer models, exploring its functionality, implementation, and significance in modern deep learning architectures.

• YouTube
• 13 minutes
• Self-Paced
• Free Video

Data Science Courses

Attention Mechanism and Self-Attention in Deep Learning - Lecture 9

Dive into attention mechanisms and self-attention concepts in deep learning, exploring their revolutionary impact on NLP and discovering practical applications in sequence models.

• YouTube
• 1 hour 18 minutes
• Self-Paced
• Free Video

Discover AI

Self-Extending LLM Context Windows Using Grouped Self-Attention

Discover how to extend LLM context windows through grouped self-attention, enabling longer text processing without model retraining. Learn implementation techniques and practical applications.

• YouTube
• 26 minutes
• Self-Paced
• Free Video

Code With Aarohi

Transformer Self-Attention - Calculating Attention Scores - LLM Series Lecture 7

Dive into scaled dot-product attention mechanics in Transformers, learning step-by-step calculation of attention scores, Q/K/V matrices, and softmax normalization for context-aware representations.

• YouTube
• 17 minutes
• Self-Paced
• Free Video

Code With Aarohi

Transformer Encoder Explained - Self-Attention, Q K V - Lecture 6

Dive into Transformer encoder mechanics, self-attention, and Q-K-V matrices with step-by-step explanations of embeddings, positional encoding, and multi-head attention fundamentals.

• YouTube
• 28 minutes
• Self-Paced
• Free Video

UofU Data Science

Pairwise Feature Importance and Effective Attention in Self-Attention Networks

Dive into advanced concepts of attention mechanisms, exploring effective attention computation, norm-based analysis, and practical applications in neural networks.

• YouTube
• 36 minutes
• Self-Paced
• Free Video

Stanford Seminar 2022 - Self Attention and Non-Parametric Transformers

Explore self-attention and Non-Parametric Transformers with PhD students discussing Transformer origins, intuitions, and recent advancements in neural network architecture.

• YouTube
• 1 hour 6 minutes
• Self-Paced
• Free Video

Data Science: Transformers for Natural Language Processing

3026 ratings at Udemy

ChatGPT, GPT-4, BERT, Deep Learning, Machine Learning & NLP with Hugging Face, Attention in Python, Tensorflow, PyTorch

• Udemy
• 19 hours 6 minutes
• Self-Paced
• Paid Course