Convolutional Network Back Propagation by Hand - The Math You Should Know

Learn the mathematical foundations of backpropagation in convolutional neural networks through detailed step-by-step calculations performed by hand. Master the multivariate chain rule and its application to computing gradients across different layers including output layers, hidden layers, pooling layers, ReLU activation functions, and most importantly, convolutional layers with feature map gradients. Work through a complete example using a specific CNN architecture, starting with a forward pass using a labeled sample, then systematically calculating gradients for each layer during the backward pass. Understand weight updates and verify your hand calculations using PyTorch code to ensure accuracy. The tutorial includes comprehensive coverage of the mathematical concepts needed for backpropagation, practical implementation details, and concludes with a quiz to test your understanding of the material.

Syllabus

00:00 What & Why back propagation?
2:01 Conv Network we will work with through the video
6:10 High level explanation
8:36 Forward pass with an labeled sample
15:44 Useful math for the backward pass multivariate chain rule
16:53 Backward pass: output layer gradients
23:44 Backward pass: 3rd weight layer gradients
26:32 Backward pass: hidden layer gradients
30:24 Backward pass: 2nd weight layer gradients
31:53 Backward pass: layer p gradients
32:27 Backward pass: pooling gradients
37:05 Backward pass: ReLU gradients
37:05 Backward pass: ReLU gradients
38:37 **IMPORTANT** Backward pass: Convolution gradients feature map gradients
47:15 Updating weights
50:20 pytorch code to verify if hand calculations are correct
51:06 Quiz Time
52:18 Summary & Conclusion