Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

CodeSignal

Sequence Models & The Dawn of Attention

via CodeSignal

Go to class Write review

Overview

You'll explore why RNNs and LSTMs struggle with long sequences, then build attention mechanisms from the ground up, mastering the QKV paradigm and creating reusable attention modules in PyTorch.

Syllabus

  • Unit 1: Revisiting Sequence Models: RNNs, LSTMs, and Their Limits
    • Building Your First LSTM Model
    • Generate Sequential Memory Challenge
    • Switching Prediction Targets
    • Training Your First LSTM Model
  • Unit 2: Introducing the Attention Mechanism
    • Building Your First QKV Tensors
    • Building Attention Score Engine
    • From Scores to Context Vector
    • Building Complex Attention Mechanisms
    • Finishing Bahdanau Attention
  • Unit 3: Scaled Dot-Product Attention and Masking in Transformers
    • Building Robust Attention Mechanisms
    • Building Attention Masks
    • Creating Attention Boundaries
    • Apply Masks to Attention Scores
  • Unit 4: Building Attention Modules
    • Building Your First Attention Module
    • Building the Attention Core
    • Implementing Attention Mask Logic
    • Complete the Attention Pipeline

Reviews

Start your review of Sequence Models & The Dawn of Attention

Go to class

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Sign up for free
Someone learning on their laptop while sitting on the floor.