CAP5415 - Training Neural Networks Part 2 - Fall 2020 - Lecture 7

Dive into advanced concepts of training neural networks in this comprehensive lecture from the University of Central Florida's CAP5415 course. Explore key topics including network parameters, learning phases, loss functions, and gradient descent optimization techniques. Gain insights into backpropagation using the chain rule, and witness a practical UCF optimization demo. Examine challenges in gradient descent, such as oscillations, and learn strategies to overcome them, including momentum and learning rate adjustments. Investigate data fitting problems, early stopping techniques, and essential training steps. Conclude with an in-depth look at AlexNet training and the innovative architecture of Residual Networks.

Syllabus

Network Parameters - recap
Learning phases - recap Images
Loss Function - recap
Train CNN with Gradient Descent
Loss Functions
Differentiability
Backpropagation - Chain Rule
UCF Optimization demo
Stochastic Gradient Descent
Gradient descent oscillations
Momentum
Lowering the learning rate
Problem of fitting
Data fitting problem
Early stopping
Training steps
AlexNet - Training
Residual Networks