Coursera Flash Sale
40% Off Coursera Plus for 3 Months!
Grab it
Learn how DigiCert, a digital security company serving 88% of the Fortune 500, built a comprehensive sales intelligence system by processing billions of certificate transparency log events using Databricks' structured streaming capabilities. Discover how this 28-minute conference talk demonstrates the implementation of a scalable data pipeline that aggregates and analyzes certificate transparency logs via public APIs to provide market and competitive intelligence. Explore the technical architecture that leverages Spark for parallel processing, structured streaming for real-time ingestion and deduplication, and Delta tables for data reliability while optimizing costs through strategic use of pools and jobs. Understand how this solution replaced reliance on third-party providers with limited data by giving DigiCert full control over their data pipeline, enabling deeper insights and automation. See how the system reliably polls public APIs in a scalable manner to fetch millions of events daily, deduplicate them, and store them in Delta tables to keep data fresh, accurate, and cost-effective. Gain insights into how this real-time data intelligence system has empowered DigiCert's sales team with actionable market intelligence, contributing directly to the company's success in securing over 28 billion web connections daily.
