Breaking the 1.5MB Barrier: Running Large Metaflow Flows with Argo for AI/ML Workloads
CNCF [Cloud Native Computing Foundation] via YouTube
AI Engineer - Learn how to integrate AI into software applications
Master Finance Tools - 35% Off CFI (Code CFI35)
Overview
Coursera Spring Sale
40% Off Coursera Plus Annual!
Grab it
Learn how to overcome etcd's 1.5MB object size limitation when running large-scale AI/ML workflows in Argo through this technical conference talk. Discover the challenges of managing extensive batch workflows for AI/ML workloads, particularly in data preparation and model training scenarios that involve numerous steps. Explore real-world examples demonstrating how this limitation creates unpredictable bottlenecks, and master the implementation of Argo's innovative solution using RDBMS offloading. Follow a detailed walkthrough of configuring the Argo offloading feature with Aurora Postgres RDS and EKS in AWS environments, enabling seamless scaling while maintaining Kubernetes compatibility.
Syllabus
Breaking the 1.5MB Barrier: Running Large Metaflow Flows with Argo for AI/ML Workloa... Saurabh Garg
Taught by
CNCF [Cloud Native Computing Foundation]