Scalable Continuous Monitoring for Large-scale A/B Experimentation

This 26-minute conference talk from Data Council features Chenyu Qiu, Staff Scientist at Uber, sharing valuable insights from building Uber's end-to-end continuous experiment monitoring platform. Discover how to overcome "peeking" problems and detect performance degradation across thousands of A/B experiments simultaneously. Gain practical knowledge about anytime-valid inference techniques, methods for eliminating noise while preserving true signals, and strategies for accelerating data-driven decisions at scale. Particularly valuable for teams scaling experimentation processes or implementing regression-adjusted estimators, this presentation offers technical solutions to common challenges in large-scale experimentation environments.