A Needle in a Haystack: Finding Threats in Billions of Cloud Logs

Discover effective strategies for identifying threats hidden within massive volumes of cloud platform audit logs in this 37-minute conference talk by Brian Davis from Red Canary. Learn how to tackle the challenge of processing over 6 billion logs per day using cloud building blocks like S3 buckets and SQS queues, as well as more advanced tools such as OpenSearch and Kubernetes. Explore techniques for creating a custom detection platform and building analytics to sift through the data deluge, transforming it into actionable alerts. Gain insights into Red Canary's approach to managing and analyzing cloud control plane logs from various platforms, including AWS CloudTrail, Google Cloud Platform Audit Logs, and Azure Activity Logs.