Simplifying Generative AI Model Training on Kubernetes using Helm Charts

Learn how to streamline generative AI model training on Kubernetes through a Helm-based approach that reduces complexity while maintaining flexibility in this 28-minute conference talk. Discover how to leverage Kubeflow Training Operators to abstract underlying complexities and create a consistent YAML interface across various training frameworks. Explore an accelerator-agnostic solution that works with multiple training technologies and examine a new Kubeflow Pipeline component for building complex, end-to-end training workflows using Helm charts. See practical demonstrations of training pipelines using Accelerate, Ray Train + Lightning, and NVIDIA's NeMo-Megatron libraries, along with automatic scaling of accelerator infrastructure using Karpenter. Gain insights into managing the diversity of frameworks, tools, and orchestration options available for AI model training on Kubernetes while preserving the innovation potential of this ecosystem.