How Bloomberg Creates a Resilient Data Analytics Platform Using Karmada

Explore how Bloomberg's Data Analytics Platform Engineering team leverages Karmada, a multi-cluster management system, to create a resilient data analytics platform in this 37-minute conference talk. Discover the challenges of deploying and managing real-time streaming, large batch ETL, and data exploration workflows using Apache Flink, Apache Spark, and Trino across multi-cluster Kubernetes environments at scale. Learn about the specific difficulties in handling varying resource requirements and uptime needs, particularly for stateful applications like Apache Flink where ensuring recovery and state conservation after downtime is critical. Examine Bloomberg's implementation of Karmada for deploying and managing Apache Flink, and understand how Karmada's capabilities can be extended to handle additional data analytics workloads including Apache Spark and Trino. Gain insights into unique requirements and real-world use cases covering resource-aware workload scheduling, custom resource requirements and health interpretation, and state conservation during application failover. Understand practical approaches to building resilient, scalable data analytics platforms in cloud-native environments through Bloomberg's proven strategies and architectural decisions.