Generative AI: Foundations and Concepts

Overview

This course provides an overview of some different concepts underpinning Generative AI, their mathematical principles, and their applications in engineering. The focus will be on the practical implementation of generative AI including, neural networks, attention mechanism, and advanced deep learning models.

Syllabus

Foundations of Neural Networks and Optimization

In this module, you will explore the foundations of neural networks, including perceptrons, architectures, and learning algorithms. You will dive deeply into optimization methods critical for efficient training, focusing on advanced techniques like Newton’s and quasi-Newton methods, momentum, RMSProp, and Adam optimization algorithms.

Regularization and Advanced Techniques

This module guides you through the mathematical approaches to regularization techniques that enhance neural network generalization and prevent overfitting. You will analyze concepts including Stein’s unbiased risk estimator, eigen decomposition, ensemble methods, dropout mechanisms, and advanced normalization techniques such as batch normalization.

Convolutional Neural Networks (CNNs)

In this module, you will examine convolutional neural networks (CNNs), including convolution operations, parameter sharing, kernel methods, and multi-dimensional data structures. You'll explore advanced CNN architectures, regularization, normalization techniques, and the implications of random kernels on network learning behavior.

Generative Models and Maximum Likelihood Learning

In this module, you will analyze the maths underpinning generative models and maximum likelihood estimation (MLE). You will explore divergence metrics such as Kullback-Leibler divergence, Bayesian network structures, and autoregressive modeling methods, focusing on their theoretical foundations and practical implications.