No More Fragile Pipelines - Kafka and Iceberg the Declarative Way

Learn how to eliminate fragile data pipelines by establishing a transparent equivalence between Kafka Topics and Iceberg Tables in this conference talk. Discover the common pain points of traditional data movement between operational systems and analytics platforms, including Kafka batching bottlenecks and Iceberg's small file problems that clutter metadata and degrade query performance. Explore Tableflow, a revolutionary approach that shifts complexity away from individual pipelines into a unified layer, enabling simpler declarative workflows. Master techniques for handling schema evolution, compaction strategies, and topic-to-table mapping while learning to continuously materialize and optimize thousands of Kafka topics as Iceberg tables. Gain practical insights for building resilient, scalable, and future-proof data architectures whether you're modernizing existing systems or starting fresh, with real-world solutions to streaming ingestion challenges and table optimization strategies.