Self-Attention Courses and Certifications

Yacine Mahdid

Masked Self-Attention from Scratch in Python

Dive into masked self-attention algorithm implementation in Python with numpy, covering theory and step-by-step coding from QKV computation to output generation, essential for understanding LLM pre-training.

• YouTube
• 14 minutes
• Self-Paced
• Free Video

Stanford Seminar 2022 - Self Attention and Non-Parametric Transformers

A seminar tracing Transformer origins and intuitions before introducing Non-Parametric Transformers.

• YouTube
• 1 hour 6 minutes
• Self-Paced
• Free Video

Aleksa Gordić - The AI Epiphany

Flash Attention 2.0 with Tri Dao - Discord Server Talks

Tri Dao explains how FlashAttention 2.0 uses IO-aware design and kernel changes to make attention more efficient for long sequences.

• YouTube
• 1 hour
• Self-Paced
• Free Video

Serrano.Academy

The Math Behind Attention Mechanisms

Uses embeddings, context similarity, and key, query, and value matrices to work through the mathematics of attention.

• YouTube
• 38 minutes
• Self-Paced
• Free Video

Donato Capitella

The Transformer Architecture: Understanding Self-Attention and Positional Encoding

Delve into the revolutionary Transformer architecture, exploring self-attention mechanisms, positional encoding, and key components that revolutionized natural language processing and machine learning.

• YouTube
• 15 minutes
• Self-Paced
• Free Video

Data Science Courses

Attention Mechanism and Self-Attention in Deep Learning - Lecture 9

Dive into attention mechanisms and self-attention concepts in deep learning, exploring their revolutionary impact on NLP and discovering practical applications in sequence models.

• YouTube
• 1 hour 18 minutes
• Self-Paced
• Free Video

Neural Breakdown with AVB

Vision Transformers: Understanding Self-Attention and Implementation in PyTorch

Dive into Vision Transformers through clear visualizations and hands-on PyTorch implementation, mastering self-attention mechanisms and comparing them with CNNs in computer vision tasks.

• YouTube
• 16 minutes
• Self-Paced
• Free Video

Neural Breakdown with AVB

Understanding Self-Attention in Transformer Models - Part 2

Dive into the mechanics and power of Self-Attention in Transformer models, exploring its functionality, implementation, and significance in modern deep learning architectures.

• YouTube
• 13 minutes
• Self-Paced
• Free Video

Code With Aarohi

Transformer Encoder Explained - Self-Attention, Q K V - Lecture 6

Dive into Transformer encoder mechanics, self-attention, and Q-K-V matrices with step-by-step explanations of embeddings, positional encoding, and multi-head attention fundamentals.

• YouTube
• 28 minutes
• Self-Paced
• Free Video

Code With Aarohi

Transformer Self-Attention - Calculating Attention Scores - LLM Series Lecture 7

Dive into scaled dot-product attention mechanics in Transformers, learning step-by-step calculation of attention scores, Q/K/V matrices, and softmax normalization for context-aware representations.

• YouTube
• 17 minutes
• Self-Paced
• Free Video

UofU Data Science

Pairwise Feature Importance and Effective Attention in Self-Attention Networks

Dive into advanced concepts of attention mechanisms, exploring effective attention computation, norm-based analysis, and practical applications in neural networks.

• YouTube
• 36 minutes
• Self-Paced
• Free Video

Transformer Models and BERT Model

126 ratings at Coursera

Learn how the Transformer architecture works, from the self-attention mechanism to the BERT model and tasks like text classification, question answering, and natural language inference.

• Coursera
• 51 minutes
• Self-Paced
• Paid Course

Core Neural Architectures & Generative Models

Implement and evaluate CNNs, sequence models, Transformers, VAEs, GANs, and diffusion pipelines across vision, temporal, NLP, and generative workflows.

• Coursera
• 10 weeks, 1 hour a week
• Self-Paced
• Paid Course

Alfredo Canziani

Self - Cross, Hard - Soft Attention and the Transformer

Explores self- and cross-attention through queries, keys, and values, then builds a Transformer encoder in PyTorch.

• YouTube
• 1 hour 12 minutes
• Self-Paced
• Free Video

Rob Gray

Classic Motor Learning Effects in Expert Performance - Focus of Attention and Self-Control

Explore the application of classic motor learning effects on expert athletes, focusing on self-controlled practice and attentional focus in basketball free throws.

• YouTube
• 14 minutes
• Self-Paced
• Free Video