Overview

This course covers practical algorithms and the theory for machine learning from a variety of perspectives. Topics include supervised learning (generative, discriminative learning, parametric, non-parametric learning, deep neural networks, support vector Machines), unsupervised learning (clustering, dimensionality reduction, kernel methods). The course will also discuss recent applications of machine learning, such as computer vision, data mining, natural language processing, speech recognition and robotics. Students will learn the implementation of selected machine learning algorithms via python and PyTorch.

Syllabus

Introduction to Statistical Learning in Engineering

This week provides an introduction to the field of statistical learning, exploring its scope and practical applications across various domains. Students will analyze how statistical learning techniques are used to make predictions, infer relationships, and uncover patterns in complex datasets. The module also offers a review of the key concepts essential for success in the course, including statistical models, data handling, and learning algorithms. By the end of the module, you will have a solid understanding of statistical learning principles and be prepared to apply them in real-world scenarios, laying the foundation for deeper exploration in machine learning and data science.

A Primer on Statistical Learning Concepts

This week introduces you to the concept of Maximum Likelihood Estimation (MLE) and its application in statistical modeling. You will gain a thorough understanding of how to mathematically implement MLE and apply it to real-world datasets. The week will revisit foundational concepts of convex optimization, offering a solid foundation in optimization techniques. Additionally, the iterative process of the gradient descent algorithm will be explored, allowing you to understand and implement this method for finding optimal solutions in machine learning models. Through a combination of theoretical knowledge and practical application, you will build essential skills in statistical estimation and optimization, preparing for advanced studies in machine learning and data analysis.

The Learning Process

In this module, you will gain a comprehensive understanding of supervised machine learning from model training to evaluation. You’ll interpret each step in the learning process and apply training and evaluation techniques to real-world data. This will enable you to fit and assess models, while addressing issues like overfitting and underfitting. By exploring the bias-variance trade-off, you can optimize models for greater accuracy and reliability. Cross-validation methods are also covered, equipping students with robust tools for model assessment and performance analysis. This week will combine theoretical insights preparing you for the advanced work in machine learning.

Linear Regression

This module, we will focus on the foundational principles of linear regression, a key technique in predictive modeling. You will learn to apply linear regression models and derive the ordinary least squares (OLS) formulation, gaining insight into how OLS is used to fit data accurately. We will also cover solution methods, including gradient descent and convex optimization, which provides a toolkit for efficient model training. You will explore regularization techniques to enhance model robustness and prevent overfitting. By implementing these regularized regression models in Python, you will gain hands-on experience in model optimization.