Learn about the inner workings and training methodology of ChatGPT in this 12-minute technical video that kicks off a series about large language models. Explore key concepts including GPT model limitations, alignment challenges, and the role of reinforcement learning in AI development. Gain detailed insights into the three-step training process of ChatGPT, with special emphasis on Reinforcement Learning from Human Feedback (RLHF). Follow along with clear explanations supported by visual demonstrations and examples of model responses, setting the foundation for understanding more advanced topics in future series installments about AI limitations and alternative tools.

Syllabus

- Intro
- Limitations of GPT models and Alignment
- Reinforcement Learning
- Reinforcement Learning from Human Feedback
- ChatGPT Model overview
- ChatGPT Model Training Step 1
- ChatGPT Model Training Step 2
- ChatGPT Model Training Step 3