Transformers from Scratch

Learn to build transformer neural networks from the ground up in this comprehensive 4-hour tutorial series. Master the fundamental components of transformer architecture by implementing self-attention mechanisms, multi-head attention, and positional encoding with hands-on coding exercises. Explore layer normalization techniques and understand how they function within transformer networks. Dive deep into both encoder and decoder architectures, breaking down their components before coding complete implementations from scratch. Build a transformer encoder in just 100 lines of code, then expand to create a full decoder architecture. Understand sentence tokenization processes and integrate all components to create a complete transformer neural network in 300 lines of code. Conclude by applying your knowledge to build a practical translator application using the transformer architecture you've constructed throughout the series.

Syllabus

Self Attention in Transformer Neural Networks (with Code!)
Multi Head Attention in Transformer Neural Networks with Code!
Positional Encoding in Transformer Neural Networks Explained
Layer Normalization - EXPLAINED (in Transformer Neural Networks)
Blowing up the Transformer Encoder!
Transformer Encoder in 100 lines of code!
Blowing up Transformer Decoder architecture
Transformer Decoder coded from scratch
Sentence Tokenization in Transformer Code from scratch!
The complete guide to Transformer neural Networks!
The complete Transformer Neural Network Code in 300 lines!
Building a Translator with Transformers