Completed
15:13 The Attention Sinks in 20B and 120B
Class Central Classrooms beta
YouTube videos curated by Class Central.
Classroom Contents
We Fine-Tuned GPT OSS 20B to Rap Like Eminem
Automatically move to the next video in the Classroom when playback concludes
- 1 0:00 Intro: Fine-tuning GPT OSS 20B-120B
- 2 4:05 The Task: Rapping in the Style of Eminem
- 3 6:15 Where we Kept the Data and the /Explicit Tag
- 4 8:08 How We Automize Fine-tuning
- 5 10:35 Understanding the Internals of the Models
- 6 15:13 The Attention Sinks in 20B and 120B
- 7 17:37 OpenAI’s New Harmony Format
- 8 23:45 Double Check your Templates
- 9 27:20 Question: Is the Harmony Format Only for Agentic Use-Cases?
- 10 28:25 How the Training Runs Went
- 11 32:40 Comparing against Llama 3.2 1B
- 12 34:10 Deployment Gotchas: How to Deploy After Training
- 13 37:48 Next Up: Supporting GPT OSS 120B
