Migrating Hybrid Big Data Platforms at Scale to Kubernetes

Explore the challenges and solutions for migrating hybrid big data platforms to Kubernetes at scale in this 37-minute conference talk by Sunil Govindan from Cloudera. Discover how Cloudera is leading the transition from legacy Hadoop to containerized Kubernetes using open-source technologies for resource management and multi-tenancy architecture. Learn about the complexities of managing stateful microservices in an enterprise-hardened multi-tenant platform through a case study highlighting the challenges of handling noisy workloads in shared Kubernetes clusters. Gain insights into concurrency-based autoscaling, latency issues in disaggregated storage from compute, and effective hierarchical quota management solutions for Kubernetes. Understand the importance of extracting accurate business insights from data consistently and the gaps in enterprise-ready big data management that hinder the transition of legacy big data workloads to containerized Kubernetes environments.