Learn Generative AI, Prompt Engineering, and LLMs for Free
Live Online Classes in Design, Coding & AI — Small Classes, Free Retakes
Overview
Google, IBM & Meta Certificates – 40% Off
One plan covers every Professional Certificate on Coursera.
Unlock All Certificates
Explore the groundbreaking Reformer model in this informative video, which addresses the resource-intensive nature of the famous Transformer architecture. Learn how the Reformer combines Locality Sensitive Hashing and concepts from Reversible Networks to significantly reduce memory usage and enable processing of much longer input sequences. Discover how this innovative approach allows for handling up to 16K tokens with just 16GB of memory, making it a game-changer for natural language processing tasks. Delve into the technical details of the model's O(LlogL) complexity, reversible residual layers, and their impact on efficiency. Gain insights into the Reformer's performance, which rivals traditional Transformer models while offering substantial improvements in memory efficiency and processing speed for long sequences.
Syllabus
Reformer: The Efficient Transformer
Taught by
Yannic Kilcher