Deep Learning for Natural Language Processing

ABOUT THE COURSE:Natural Language Processing finds its applications nearly everywhere -- be it Machine Translation,Question answering, Text Summarization, Dialogs, etc. In the last decade, Deep Learning basedmethods have given very good performance across a variety of NLP tasks, and have become a default choice for NLP problems. This course aims to give a thorough understanding of various deep learning architectures along with their specific use-cases in NLP.The course will also introduce the fundamental ideas behind training as well as fine-tuning/prompting the Large Language Models, which include in-context-learning, Parameter-efficient-fine-tuning,Reinforcement Learning through Human Feedback (RLHF). The course will also offer hands-on tutorials to help students master this subject.INTENDED AUDIENCE: Advanced UG and PG studentsPREREQUISITES: Participants should have done a course on MachineLearning, and should know basics of PythonProgrammingINDUSTRY SUPPORT: Google,Microsoft, Amazon, Flipkart, Adobe

Syllabus

Week 1:

Introduction to NLP: What is Natural LanguageProcessing? A brief primer on word and sentence level tasks and n-gram language Model.

Week 2:Introduction to Deep Learning

Shallow and Deep Neural Networks
Representation Learning

Week 3:Word Representations

Word2Vec
Glove
fastText,
Multilingualrepresentations with emphasis on Indian Languages

Week 4:Recurrent Neural Networks

RNN LMs
GRUs, LSTMs, Bi-LSTMs
LSTMs for Sequence Labeling
LSTMs for Sequence to Sequence

Week 5:Attention Mechanism

Sequence to Sequence with Attention
Transformers: Attention is all you need

Week 6:Self-supervised learning (SSL), Pretraining

Designing SSL objectives
Pretrained Bi-LSTMs: ELMO
Pretrained Transformers: BERT, GPT, T5, BART

Week 7:

Applications: Question Answering, Dialog Modeling, TextSummarization
Multilingual extension with application to Indian languages

Week 8:Instruction Fine-tuning, FLAN-T5, Reinforcement Learningthrough Human Feedback (RLHF)
Week 9:In-context learning, chain-of-thought prompting. ScalingLaws. Various Large Language Models and unique architectural differences
Week 10:Parameter Efficient Fine-tuning (PEFT) - LoRA, QLoRA
Week 11:Handling Long Context, Retrieval Augmented Generation(RAG)
Week 12:Analysis and Interpretability, ethical considerations