Deep Learning: Model Optimization and Tuning

Introduction

• Optimizing neural networks
• Setting up exercise files

1. Introduction to Deep Learning Optimization

• Review of artificial neural networks
• An artificial neural network (ANN) model
• Model optimization and tuning
• The deep learning tuning process
• Experiment setups for the course

2. Tuning the Deep Learning Network

• Epoch and batch size tuning
• Epoch and batch size experiment
• Hidden layers tuning
• Determining nodes in a layer
• Choosing activation functions
• Initializing weights

3. Tuning Back Propagation

• Vanishing and exploding gradients
• Batch normalization
• Optimizers
• Optimizer experiment
• Learning rate
• Learning rate experiment

4. Overfitting Management

• Overfitting in ANNs
• Regularization
• Regularization experiment
• Dropouts
• Dropout experiment

5. Model Tuning Exercise

• Tuning exercise: Problem statement
• Acquire and process data
• Tuning the network
• Tuning backpropagation
• Avoiding overfitting
• Building the final model