Garbage in Garbage Out - Generating Useful Log Data in Complex Environments

Explore effective log data generation techniques for complex environments in this 51-minute conference talk from Derbycon 2016. Learn why logging everything is ineffective and discover the importance of meaningful log data. Delve into using thresholds, frequency analysis, and noise balancing to improve log quality. Examine log management systems, including ingestion, parsing, and visualization. Gain insights into JSON standards, Python integration, and handling old services and web server logs. Follow along with a demo using Phobos and Squid, and see real-world scenarios of identifying proxy abuse. Understand how to set up alerts, trend on thresholds, and utilize tools like Splunk. Discover the benefits of good, reliable data for small companies and large enterprises alike.