Learn how live systems reshape the data they learn from and why the signals you measure often become the unintended goal. You’ll design oversight, escalation paths, and monitoring triggers to manage systems in production. Challenge the assumption that rising scores equal success by learning to spot drift, unintended optimization, and feedback loops in automated decisions.
Overview
Syllabus
- Unit 1: Choosing Better System Signals
- Matching Signals That Actually Matter
- Choosing Objectives Beyond Click Counts
- Fixing the Signal Before the Build
- Unit 2: Tracing How Outcomes Feed Back Into Later Decisions
- Mapping Loops and Scaling Boundaries
- Spotting Loops and Reward Misalignment
- Challenging a Self-Confirming Trend
- Unit 3: Monitoring Drift and Unintended Optimization
- Matching the Four Weekly Evidence Types
- Diagnosing a Rising Score
- Facing the Rising Score
- Unit 4: Establishing Oversight, Escalation, and Accountability
- Deciding When Humans Must Decide
- Matching Incidents to Escalation Responses
- Securing the Go/No-Go Decision