Metadata-Driven Streaming Ingestion Using DLT, Azure Event Hubs and a Schema Registry

Learn how to implement metadata-driven streaming data ingestion using Delta Live Tables (DLT), Azure Event Hubs, and Schema Registry in this 30-minute conference talk. Discover Plexure's journey from batch processing to near real-time streaming for ingesting hundreds of millions of customer activities and transactions daily. Explore how this transformation enables immediate reaction to customer behavior rather than waiting hours or days, powering personalization engines and marketing campaign insights. Understand how leveraging a Schema Registry creates a metadata-driven framework that allows data producers to evolve schemas confidently while ensuring downstream processes continue running smoothly, and seamlessly publish new datasets without requiring Data Engineering assistance. Watch a live demonstration of the end-to-end process and learn about DLT meta-programming implementation techniques for building production-ready streaming data pipelines.