On-Demand Container Partitioning for Distributed ML

Learn about 2DFS, a revolutionary two-dimensional filesystem designed to address critical limitations in container-based distributed machine learning deployments through this 19-minute conference presentation from USENIX ATC '25. Discover how researchers from Technical University of Munich and TU Delft tackle the inefficiencies of current container filesystem architectures when handling dynamic ML model retraining, partitioning, and updates. Explore the complete 2DFS ecosystem including its builder, registry, and cache hierarchy components that streamline ML model build and deployment processes. Examine comprehensive evaluation results from 14 real-world ML models demonstrating 2DFS's impressive performance gains: up to 56x faster build times, 25x improved caching efficiency, and on-demand image partitioning capabilities with negligible overhead. Understand how this OCI-compliant solution seamlessly integrates with existing container infrastructures and workflows, making it a practical advancement for distributed ML deployments at scale.