Open Pretrained Transformer - ML Coding Series

Intro - open pretrained transformer
Setup creating the cond env
Setup patch the code
Collecting train script arguments
Training script walk-through
Constructing a dummy task
Building the transformer model
CUDA kernels C++ code
Preparing a dummy dataset
Training loop
Zero grad loss scaling
Forward pass through a transformer
IMPORTANT loss, scaling, mixed precision, error handling
Outro