Coursera Flash Sale
40% Off Coursera Plus for 3 Months!
Grab it
Learn how Q6 Cyber transformed their e-crime and fraud intelligence architecture by placing Apache Pulsar at the center of their data platform in this keynote presentation. Discover the journey from a legacy system built around home-grown Java applications, Google's PubSub, Google Cloud Storage, and self-hosted OpenSearch clusters to a modern, Pulsar-centric architecture. Explore the challenges encountered during this architectural transformation, including serialization issues, multi-threading optimization for performance, function execution and scaling, and properly scaling both off-the-shelf and custom sources and sinks. Understand how Pulsar enhanced flexibility in routing, processing, and storing real-time actionable intelligence data, enabling faster delivery of more accurate intelligence products to clients. Examine the implementation of streaming new data into a Hudi-based data lakehouse and learn about plans to migrate an existing repository of 75 billion objects using StreamNative features for streamlined reporting and historical analysis. Gain insights into the growing pains, lessons learned, and solutions developed while transitioning from architectural constraints to innovation-focused development in the cybersecurity intelligence domain.
