Next-Gen Data Infrastructure - Building Resilient, Scalable Architecture with Apache Pulsar

Learn how to build resilient, scalable data infrastructure using Apache Pulsar through a real-world case study from Blueshift's engineering team. Discover how to eliminate cascading failures by replacing synchronous RPC chains with fully asynchronous Pulsar-based messaging, and explore techniques for isolating tenants and workloads using thousands of lightweight Pulsar topics to ensure gold-tier SLAs for critical updates while separating low-priority traffic. Master the implementation of streamlined archival with low-cost, high-throughput pipelines that handle massive data volumes effortlessly, and understand how to scale horizontally on demand by adding client-specific capacity during high-traffic periods without downtime. Explore how to simplify data architecture with multi-subscription fan-out using a single durable topic to feed multiple downstream services without duplication. Gain insights into async communication excellence through Pulsar's asynchronous model for reduced latency and decoupled services, SLA-driven topic design for enforcing QoS guarantees efficiently, multi-tenant resilience through quota management and throttling, archival at scale with efficient streaming-to-storage pipelines, and operational best practices including monitoring, capacity planning, and troubleshooting Pulsar in production environments.