Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

Foundations of Transformer Architectures for Natural Language Processing

Packt via Coursera

Go to class Write review

Details

Go to class

Provider

Coursera
Pricing

Paid Course
Languages

English
Certificate

Certificate Available
Effort

6 weeks, 1 hour a week
Sessions

Self-Paced
Level

Intermediate
Subtitles

English

Part of

Transformers for NLP and Computer Vision

Overview

Google, IBM & Meta Certificates – 40% Off

One plan covers every Professional Certificate on Coursera.

Unlock All Certificates

Explore the core principles and foundational architectures of transformers, focusing on their revolutionary impact on natural language processing. Gain a deep understanding of how transformer models work and the tasks they enable. This course introduces the fundamental concepts behind transformer models, tracing their evolution and examining their architecture in detail. Learners will discover how transformers have transformed natural language processing, from basic input representations to advanced tasks such as reading comprehension and translation. By the end of the course, you will be equipped to understand and evaluate transformer-based models and their applications in NLP. Through a blend of clear explanations, real-world examples, and guided explorations, this course builds your understanding of transformer models step by step. You will progress from foundational concepts to practical applications, ensuring a solid grasp of both theory and practice. This course is part one of a three-course Specialization designed to build a complete and cohesive understanding of the subject. While it offers valuable skills on its own, you'll gain the most benefit by progressing through all three courses as a structured learning journey. This course is based on Transformers for Natural Language Processing and Computer Vision, by Denis Rothman. Packt is one of the world's most prolific publishers of cutting-edge technical content. For over two decades we've made it our mission to curate and publish the knowledge of only the very best technical experts. We focus on real-world courses that help our customers get the job done, with coverage that extends across a wide range of established and cutting-edge technical topics. If you're an individual or an organisation that embraces learning by doing, Packt is the perfect fit for you.

Syllabus

What are Transformers?

This module introduces the evolution and impact of transformer models in artificial intelligence, highlighting their foundational role in modern AI applications. Learners will explore the history, architecture, and practical deployment of transformers, including available cloud and API resources. The module also examines the changing responsibilities of AI professionals in a rapidly evolving technological landscape.

Getting Started with the Architecture of the Transformer Model

This module introduces the foundational components of the Transformer model, including self-attention, multi-head attention, input embeddings, and positional encoding. Learners will explore how these elements interact to process language data and understand the role of normalization in model architecture. Practical exercises guide students through implementing and analyzing key sublayers of the Transformer.

Emergent vs Downstream Tasks: The Unseen Depths of Transformers

This module delves into how transformer models tackle complex Natural Language Understanding (NLU) tasks, including multi-sentence comprehension and coreference resolution. Learners will examine performance metrics, human baselines, and advanced benchmarks like SuperGLUE to understand the evolving capabilities of AI. Real-world examples such as MultiRC and the Winograd Schema Challenge illustrate the depth and challenges of emergent and downstream tasks.

Advancements in Translations with Google Trax, Google Translate, and Gemini

This module introduces modern machine translation techniques using tools like Google Trax, Google Translate, and Gemini Transformers. Learners will preprocess translation datasets, implement evaluation metrics such as BLEU scores, and explore the practical applications of transformer-based models in multilingual contexts.

Diving into Fine-Tuning through BERT

This module guides learners through the process of fine-tuning BERT models for natural language processing tasks using Hugging Face. You will explore key concepts such as tokenization, next-sentence prediction, optimizer configuration, and evaluation metrics like MCC. By the end, you'll gain practical skills in adapting pretrained transformer models to specific NLP problems.

Pretraining a Transformer from Scratch through RoBERTa

This module guides learners through the process of building and pretraining a custom RoBERTa-based transformer model using Hugging Face tools. Participants will gain hands-on experience with tokenizer training, dataset preparation, model parameter exploration, and practical pretraining for NLP applications such as generative AI customer support.