MLOps Essentials: Monitoring Model Drift and Bias

Introduction

• The need for model monitoring
• Setting up the exercise files

1. Introduction to Model Monitoring

• ML models in production
• Challenges with serving models in production
• Metrics to monitor
• Data for model monitoring

2. Model Drift Basics

• Introduction to model drift
• Concept drift
• Feature drift
• What causes drift?
• Drift remediation process

3. Detecting Model Drift

• Detecting concept drift
• Concept drift detection example
• Detecting feature drift
• Feature drift detection example
• Detecting drift in text and images
• Software for drift detection

4. Drift Monitoring Process and Best Practices

• Drift monitoring pipeline
• Analyzing drift trends
• Discovering root causes for drift
• Retraining to overcome drift

5. Introduction to Model Bias

• Fairness and bias
• Fairness in ML
• Sources of ML bias
• Protected attributes
• Demographic parity

6. Bias Detection and Best Practices

• Bias detection techniques
• Equal opportunity score
• EOS example
• Bias detection software
• Overcoming bias in ML

Conclusion

• Next steps