Natural Language Processing - Spring 2024

This course introduces natural language processing from tokenization, vector semantics, and machine learning basics through transformers, language modeling, fine-tuning, and major NLP tasks. Topics include translation, tagging, parsing, semantic role labeling, question answering, multilingual and multimodal language models, and summarization.

Syllabus

Intro to NLP; Course logistics
Machine learning basics; Perceptron
Logistic regression; Tokenization
Vector semantics; Embeddings
Neural network basics
Language modeling
Neural machine translation
Attention; Transformer (Encoder)
Transformer (Decoder); Pretraining & Finetuning
Finetuning DeBERTa in (demo); Instruction FT
Finetuning a seq2seq model in (demo); RLHF
“Taskification” and Valid Benchmarks
Midterm overview
POS Tagging; Named Entity Recognition; Hidden Markov Model
Hidden Markov Model: Parameter Estimation & Efficient Inference
Constituency parsing
Dependency parsing
Semantic parsing; SRL
Coreference resolution
Probing
Question Answering
Multilingual and multimodal LLMs
Topics in Responsible AI
Final overview
Summarization