How DreamWorks Animation Orchestrates Massive Workloads for Films with HTCondor

Discover how DreamWorks Animation manages massive computational workloads in this conference talk from Ubuntu Summit 2024. Explore the sophisticated implementation of HTCondor, an open-source workload orchestration and scheduling system, through the expertise of Mark Jackels, Senior Director of Production Software at DreamWorks Animation. Learn about the integration of essential open-source technologies including Grafana, InfluxDB, PostgreSQL, and Greenplum for monitoring and optimizing large-scale processing in animated film production. Gain insights into how DreamWorks maintains over 95% utilization of their render farm, ensuring efficient production of beloved animated features. Drawing from his extensive experience across aerospace, automotive, and commercial industries since 2002, Jackels shares his knowledge in managing teams focused on large-scale applications, distributed computing, data pipelines, and visualizations, including recent innovations in machine learning integration for enhanced imaging techniques, quality, and workflow optimization in film production.