Behind the Cloud at Twitter - Capacity Management and Resource Optimization for Large-Scale Aurora/Mesos Clusters

Learn how to implement capacity management solutions for large-scale compute clusters through this 31-minute conference talk from DevOpsDays Tel Aviv. Discover the technical approach used to increase existing Aurora/Mesos cluster capacity by 9%, resulting in nine-figure CAPEX savings over multiple years while addressing inefficiency challenges at data center scale. Explore methods for identifying underutilized resources including CPU, memory, and network within storage platforms, and understand how to implement modern solutions based on experience with Solaris Zones, Cgroups, and Namespaces. Gain insights into colocation strategies for compute and storage workloads on bare-metal machines, and learn about the capacity management and demand forecasting challenges faced when managing one of the world's largest compute clusters during Twitter's transition to Kubernetes re-platforming.