Neural Nets for NLP 2021 - Structured Prediction with Local Independence Assumptions

This lecture explains structured prediction in natural language processing, focusing on local independence assumptions, conditional random fields, and BiLSTM-CRF sequence labeling. It also covers exposure bias, normalization, training, decoding, and dynamic programming.

Syllabus

CS11-747 Neural Networks for NLP
A Prediction Problem
Types of Prediction
Why Call it "Structured" Prediction?
Many Varieties of Structured Prediction!
Why Model Interactions in Output? . Consistency is important! time flies like an arrow
Sequence Labeling w
Recurrent Decoder
Teacher Forcing and Exposure Bias
An Example of Exposure Bias
Models w/ Local Dependencies
Local Normalization vs. Global Normalization
Conditional Random Fields
Potential Functions
BILSTM-CRF for Sequence Labeling
CRF Training & Decoding
Forward Calculation Middle Parts
Forward Calculation: Final Part • Finish up the sentence with the sentence final symbol
Revisiting the Partition Function
Training Details
Generalized Dynamic Programming Models • Decomposition Structure: What structure to use, and thus also what dynamic programming to perform? . Featurization: How do we calculate local scores?