Coursera Flash Sale
40% Off Coursera Plus for 3 Months!
Grab it
Explore the critical need for stream processor evolution in this 37-minute conference talk that examines why current streaming solutions like Kafka Streams must adapt to meet the demands of modern, large-scale streaming workloads. Discover how the explosive growth of Kafka adoption over the past decade has outpaced the development of stream processing tools, creating operational challenges for companies deploying streaming infrastructure at unprecedented scales. Learn about the architectural changes required for next-generation stream processors, including decoupling compute from storage, leveraging Kubernetes-native designs, and utilizing powerful building blocks like object storage to enhance operational management. Understand the importance of matching appropriate stream processing tools to specific use cases and embracing modular, LEGO-like architectures that allow developers to assemble components tailored to their application needs. Gain insights into the future of stream processing, particularly as AI and real-time decision-making applications demand new levels of scalability and multi-tenancy that existing solutions may not adequately support. Examine current challenges in observability for Kafka stream processors and the need for comprehensive data flow visibility in complex, interconnected systems. Consider how platform teams versus application teams should approach stream processor management based on criticality and use case requirements, and explore the potential timeline for stream processors capable of supporting AI-scale applications.
