Decision Trees and Ensemble Methods

This course introduces decision trees and develops them into ensemble methods. It covers CART, impurity, regularization, bias-variance tradeoffs, ExtraTrees, bagging, random forests, feature importance, boosting, and major implementations including scikit-learn, XGBoost, LightGBM, and CatBoost.

Syllabus

DTEL1 - 1 Welcome to DTE class.
DTEL1 - 2 Class Introduction.
DTEL1 - 3 Introduction to Decision Trees.
DTEL1 - 4 Impurity Functions.
DTEL1 - 5 CART Algorithm.
DTEL1 - 6 Basic Properties of Decision Trees.
DTEL1 7 Basic Regularization of Trees.
DTEL1 - 8 Sklearn Trees.
DTEL1 - 9 Conclusion.
DTEL2 2 1 Introduction.
DTEL2 2 2 Bias variance trade off.
DTEL2 2 3 Bias variance Decomposition.
DTEL2 2 4 Generalizations of bias variance tradeoff.
DTEL2 2 5 ExtraTrees Algorithm.
DTEL2 2 6 ExtraTrees with Sklearn.
DTEL2 2 7 Conclusion.
DTEL3 3 1 Introduction.
DTEL3 3 2 Bootstrap.
DTEL3 3 3 Bagging.
DTEL3 3 4 1 Example.
DTEL3 3 5 Random Forest.
DTEL3 3 4 2 Example Notebook.
DTEL3 3 6 General Ensembling.
DTEL4 4 1 Introduction.
DTEL4 4 2 Proximities.
DTEL4 4 3 Proximities Visualizations.
DTEL4 4 4 Feature Importance's.
DTEL4 4 5 Limitations of Tree Feature Importance.
DTEL4 4 6 Feature Importance's in Random Forest.
DTEL4 4 7 Summary.
DTEL5 5 1 Introduction.
DTEL5 5 2 Boosting.
DTEL5 5 3 Gradient Boosting.
DTEL5 5 4 XGBoost.
DTEL5 5 5 LightGBM.
DTEL5 5 6 CatBoost.