Coursera Flash Sale
40% Off Coursera Plus for 3 Months!
Grab it
This conference talk from SREcon25 Americas explores a cost-effective Elasticsearch architecture designed to scale beyond petabytes of data. Learn how Leonardo Antônio dos Santos from Workday, Inc. implemented a "Cluster of Clusters" design that efficiently manages tens of petabytes of Elasticsearch data while processing trillions of indexed logs monthly. Discover practical strategies for optimizing ingestion, search, and cross-cluster search traffic through a centralized management approach with standardized data clusters. The presentation covers the implementation of a custom cluster health service based on the USE Method for intelligent query routing, real-time auditing for problematic query detection, and automated rate-limiting for high-demand users. See how these innovations reduced compute costs by 57%, achieved significant storage savings, and enhanced both scalability and migration efficiency. Gain valuable insights, benchmarks, and real-world examples to help optimize large-scale Elasticsearch deployments while maintaining performance and controlling costs.
