Explore comprehensive machine learning fundamentals through this lecture series covering supervised learning, decision trees, linear models, computational learning theory, and advanced algorithms. Begin with foundational concepts including the supervised learning setup and decision tree construction, then progress through linear classifiers, perceptron algorithms, and linear regression. Delve into computational learning theory with PAC learning frameworks, Occam's Razor principles, and VC dimension analysis. Master advanced techniques including boosting algorithms, ensemble methods, support vector machines with stochastic gradient descent optimization, and loss minimization strategies. Examine Bayesian learning approaches, maximum likelihood estimation for regression, and logistic regression methods. Conclude with neural network architectures, backpropagation algorithms, practical implementation considerations, and guidance for building real-world machine learning applications, while exploring the distinction between generative and discriminative learning paradigms.

Syllabus

Machine Learning: Lecture 1: Introduction & Course Information
Machine Learning: Lecture 2: Supervised Learning: The setup
Machine Learning: Lecture 3a: Supervised Learning - The Setup (continued)
Machine Learning - Lecture 3b: Decision Trees
Machine Learning: Lecture 4: Learning decision trees
Machine Learning: Lecture 5: Decision trees (continued)
Machine Learning: Lecture 6a: Decision trees (continued)
Machine Learning: Lecture 6b: Linear Models
Machine Learning: Lecture 7a: Linear Classifier Expressiveness
Machine Learning: Lecture 7b: How good is a learning algorithm?
Machine Learning: Lecture 7c: Online Learning
Machine Learning: Lecture 8: Mistake Bound Learning
Machine Learning: Lecture 9a: On representations and learning
Machine Learning: Lecture 9b: Perceptron
Machine Learning: Lecture 10: Perceptron mistake bound
Machine Learning: Lecture 11: Linear Regression
Machine Learning: Lecture 12a: Introduction to Computational Learning Theory
Machine learning: Lecture 12b: PAC learning introduction
Machine learning: Lecture 13a: PAC learning
Machine Learning: Lecture 13b: Occam's Razor for consistent learners
Machine Learning: Lecture 14a: Occam's Razor (continued)
Machine Learning: Lecture 14b: Positive and Negative Learnability Results
Machine Learning: Lecture 15a: Positive and Negative Learnability Results
Machine Learning: Lecture 16: Agnostic Learning
Machine learning: Lecture 17: Shattering
Machine Learning: Lecture 18a: The VC Dimension
Machine Learning: Lecture 18b: Boosting
Machine Learning: Lecture 19: Boosting & Ensembles
Machine Learning: Lecture 20: Support Vector Machines
Machine learning: Lecture 21a: SVMs (continued)
Machine learning: Lecture 21b: Stochastic Gradient Descent for SVMs
Machine Learning: Lecture 22a: SGD for SVMs
Machine Learning: Lecture 22b: Loss Minimization
Machine Learning: Lecture 23: Bayesian Learning
Machine Learning: Lecture 24a: Maximum Likelihood Estimation for Regression
Machine Learning: Lecture 24b: Logistic regression
Machine Learning: Lecture 25: Neural Networks
Machine Learning: Lecture 26: Backpropagation
Machine Learning: Lecture 27a: Practical issues with Neural Networks
Machine Learning: Lecture 27b: Generative & Discriminative learning
Machine Learning: Lecture 28: Practical advice for building machine learning applications