Neural Nets for NLP - Models with Latent Random Variables

This lecture explains generative and discriminative models with latent random variables, focusing on variational autoencoders and their NLP applications. Topics include variational inference, reconstruction loss, KL regularization, reparameterization, discrete latent-variable sampling, and methods for improving latent representations.

Syllabus

Introduction
Discriminative vs generative
Observed vs latent variables
Quiz
Latent Variable Models
Types of latent random variables
Example
Loss Function
Variational inference
Reconstruction loss and kl regularizer
Regularized auto encoder
Regularized autoencoder
Learning the VAE
Reparameterization Trick
General
Language
VAE
Reparameterization
Motivation
Consistency
Semantic Similarity
Solutions
Free Bits
Weaken Decoder
Aggressive Inference Network
Handling Discrete latent variables
Discrete latent variables
Sampling discrete variables
Gumball softmax
Application examples
Discrete random variables
Tree structured latent variables
Discussion question