Transfer Learning Larochelle

Explore transfer learning in deep learning with Hugo Larochelle's 44-minute conference talk from KDD2020. Delve into the importance of transfer learning, few-shot learning, and modern fine-tuning techniques. Examine prototypical networks, model-agnostic meta-learning, and the concept of training a fine-tuning procedure. Evaluate the meta-dataset approach and investigate universal representations and their transformers. Gain insights into the latest results in the field and discuss remaining challenges in transfer learning for deep neural networks.

Syllabus

Intro
WHYTRANSFER LEARNING • Deep leaming successes have required a lot of labeled training data
FEW-SHOT LEARNING
MODERN TRANSFER LEARNING: FINE-TUNING
LEARNING PROBLEM STATEMENT
PROTOTYPICAL NETWORKS
MODEL-AGNOSTIC META-LEARNING • Training a 'fine-tuning procedure
EVALUATION: META-DATASET
UNIVERSAL REPRESENTATIONS
UNIVERSAL REPRESENTATION TRANSFORMER
RESULTS
REMAINING CHALLENGES