Completed
Unigram Tokenization
Class Central Classrooms beta
YouTube videos curated by Class Central.
Classroom Contents
Hugging Face Course - Fast Tokenizers and Token Classification Pipelines - Chapter 6
Automatically move to the next video in the Classroom when playback concludes
- 1 Why are fast tokenizers called fast?
- 2 Fast tokenizer superpowers
- 3 Inside the Token classification pipeline (PyTorch)
- 4 Inside the Token classification pipeline (TensorFlow)
- 5 Inside the Question answering pipeline (PyTorch)
- 6 Inside the Question answering pipeline (TensorFlow)
- 7 Training a new tokenizer
- 8 What is normalization?
- 9 What is pre-tokenization?
- 10 Byte Pair Encoding Tokenization
- 11 WordPiece Tokenization
- 12 Unigram Tokenization
- 13 Building a new tokenizer
