Beyond TSDB - Unlocking Prometheus with Parquet for Modern Scale

Explore how Apache Parquet can revolutionize Prometheus storage architecture in this 29-minute conference talk from CNCF. Learn from engineers at Grafana Labs, AWS Managed Prometheus, and Cloudflare as they share their collaborative efforts to replace the traditional Prometheus TSDB format with Parquet's columnar storage approach. Discover the architectural decisions, production challenges, and performance improvements achieved when storing labels and chunks in Parquet format. Understand how this innovation addresses critical limitations with high cardinality metrics and slow queries from object storage, particularly relevant as OpenTelemetry drives increased time series dimensions. Gain insights into real-world production experiences, including specific problems encountered and solutions implemented across Mimir, Cortex, and Thanos projects. See how Parquet's big data analytics capabilities can solve longstanding Prometheus scalability issues and enable the next evolution of cloud-native monitoring. Learn about the open-source collaborative approach used throughout this development, with work conducted transparently through CNCF Slack and Prometheus Community GitHub, and discover how you can contribute to bringing Parquet support directly into Prometheus core.