Latent Variable EBMs for Structured Prediction

This lecture examines latent-variable energy-based models for structured prediction, covering training methods, contrastive and non-contrastive learning, generative adversarial networks, factor graphs, Viterbi decoding, and graph transformer networks.

Syllabus

– Welcome to class
– Training of an EBM
– Contrastive vs. regularised / architectural methods
– General margin loss
– List of loss functions
– Generalised additive margin loss
– Joint embedding architectures
– Wav2Vec 2.0
– XLSR: multilingual speech recognition
– Generative adversarial networks GANs
– Mode collapse
– Non-contrastive methods
– BYOL: bootstrap your own latent
– SwAV
– Barlow twins
– SEER
– Latent variable models in practice
– DETR
– Structured prediction
– Factor graph
– Viterbi algorithm whiteboard time
– Graph transformer networks
– Graph composition, transducers
– Final remarks