Recurrent Neural Networks, Vanilla and Gated - LSTM

This course introduces vanilla and gated recurrent neural networks through sequence-to-sequence architectures, backpropagation through time, language modeling, and vanishing or exploding gradients. It demonstrates LSTM sequence classification and activation inspection using PyTorch and Jupyter Notebook.

Syllabus

– Good morning
– How to summarise papers as @y0b1byte with Notion
– Why do we need to go to a higher hidden dimension?
– Today class: recurrent neural nets
– Vector to sequence vec2seq
– Sequence to vector seq2vec
– Sequence to vector to sequence seq2vec2seq
– Sequence to sequence seq2seq
– Training a recurrent network: back propagation through time
– Training example: language model
– Vanishing & exploding gradients and gating mechanism
– The Long Short-Term Memory LSTM
– Jupyter Notebook and PyTorch in action: sequence classification
– Inspecting the activation values
– Closing remarks