Overview
AI, Data Science & Cloud Certificates from Google, IBM & Meta — 40% Off
One plan covers every Professional Certificate on Coursera. 40% off Coursera Plus Annual.
Unlock All Certificates
Explore a user-friendly approach to working with transformers and large language models for natural language processing.
Syllabus
Introduction
- Learning about Large Language Models
- What are large language models?
- Transformers in production
- Transformers: History
- Transfer learning
- Transformer: Architecture overview
- Self-attention
- Multi-head attention and Feed Forward Network
- GPT-3
- GPT-3 use cases
- Challenges and shortcomings of GPT-3
- GLaM
- Megatron-Turing NLG Model
- Gopher
- Scaling laws
- Chinchilla
- BIG-bench
- PaLM
- OPT and BLOOM
- GitHub models
- Accessing Large Language Models using an API
- Inference time vs. pre-training
- Going further with Transformers
Taught by
Jonathan Fernandes