Perception Data Pipeline Reconstruction With Argo Workflows

Learn how to reconstruct large-scale perception data pipelines using Argo Workflows in this 23-minute conference talk from CNCF. Discover why GAC R&D Center chose Argo Workflows as their core orchestration solution for autonomous driving data perception systems that process over 40,000 tasks daily. Explore the implementation of end-to-end data perception platforms that doubled task execution efficiency through strategic pipeline design across multiple scenarios including data preprocessing, simulation, and offline inference. Gain insights into practical challenges and solutions for managing large-scale data perception pipelines, with detailed coverage of task concurrency optimization strategies, file transfer acceleration techniques, automatic GPU failure retry mechanisms, GPU utilization improvements, and performance enhancement methods. Understand the complexities of orchestrating high-concurrency workflows in autonomous driving research environments and learn proven approaches for scaling perception data processing operations effectively.