Taming Dependency Chaos for LLM in Kubernetes

Learn how to solve Python dependency management challenges when deploying Large Language Models (LLMs) in Kubernetes environments in this conference talk from KubeCon + CloudNativeCon. Discover the common pain points AI developers face, including the nightmare of maintaining base images with rapidly evolving AI packages, productivity blocks from manual pip installations, and costly GPU time wasted on package downloads. Explore a comprehensive solution using the BaizeAI dataset project that introduces a Custom Resource Definition (CRD) to describe dependencies and environments, Kubernetes Jobs for pre-loading packages, Persistent Volume Claims (PVC) for storage and mounting, conda for environment switching, and cross-namespace sharing capabilities. Gain practical insights into streamlining your AI development workflow and eliminating dependency chaos in cloud-native LLM deployments.