From LV-EBM to Target Prop to Autoencoder

This course develops autoencoders from target propagation, covering reconstruction costs, under- and overcomplete hidden layers, denoising and contractive autoencoders, and variational autoencoders.

Syllabus

– 2021 edition disclaimer
– Conditional and unconditional LV EBM
– Variables' name: x, y, z, h, ỹ
– LV EBM training recap, warm case
– LV EBM training recap, zero-temperature limit
– Today's plan: the missing step
– Target propagation
– From target prop to autoencoder
– Reconstruction costs
– Loss functional
– Under and over complete hidden layer
– Denoising autoencoder
– Contractive autoencoder
– Autoencoders recap
– From autoencoder to variational autoencoder
– Comparison between variational autoencoder and denoising autoencoder
– How a variational autoencoder actually works
– The bubble-of-bubble variational autoencoder interpretation
– And that was it :