Lessons Learned from Performance Monitoring in Chrome - Performance Metrics and Benchmarking

Explore lessons learned from Chrome's performance monitoring in this conference talk from performance.now() 2019. Dive into Annie Sullivan's extensive experience with performance metrics and monitoring, gaining insights applicable to web site performance monitoring. Discover effective strategies for performance metric and benchmark design, techniques for managing benchmark noise in lab environments, and methods for interpreting the nuances of Real User Monitoring (RUM) data. Learn about properties of good top-level metrics, key moments in page load, and the challenges of accurately measuring main content loading. Examine the pros and cons of lab testing, A/B testing, and RUM, while exploring best practices for controlled experimentation and improving reproducibility in test environments. Gain valuable takeaways to enhance your approach to web performance monitoring and optimization.

Syllabus

Intro
Agenda for our trip
Properties of a Good Top Level Metric
Key Moments in Page Load
Goal: When is Main Content Loaded?
Main Content Painted Metric Priorities
Brainstorm Simple Metrics
And Test Them for Accuracy
What About Splash Screens?
What About Background Images?
What is a "Text Paint"?
What About User Actions During Load?
Validating using HttpArchive
Monitoring stages
Lab Testing Pros and cons
Benchmark Realism: Real-world Stories
Better Reproducibility: Test Environment
Better Reproducibility: Diagnostic Metrics
Better Reproducibility: Reference Runs
Better Reproducibility: Change Detection
A/B Testing Pros and cons
Should be "Controlled Experimentation"
Best practices for experiment groups
Real User Monitoring
Metric Breakdowns
Make RUM more like A/B Testing
Takeaways