CNCF [Cloud Native Computing Foundation] Courses

How We Save $900 per Day with Self-Hosted AI - Building Scalable Local LLM Infrastructure

Discover how to cut AI costs by 90% through self-hosted LLM infrastructure, exploring private cloud deployment strategies and GPU orchestration for scalable AI applications.

• YouTube
• 13 minutes
• Self-Paced
• Free Video

Multi-Node Finetuning LLMs on Kubernetes: A Practitioner's Guide

Master practical techniques for multi-node LLM finetuning on Kubernetes, covering cluster preparation, optimization strategies, and network topology comparisons for enhanced model performance with PyTorch FSDP.

• YouTube
• 27 minutes
• Self-Paced
• Free Video

Scaling Private LLM Model Services with Kserve and Modelcar OCI - A Real-World Implementation

Discover how to efficiently deploy and scale private LLM services using Kserve and Modelcar OCI in Kubernetes, focusing on optimized GPU utilization and streamlined artifact delivery for enhanced MLOps.

• YouTube
• 37 minutes
• Self-Paced
• Free Video

Introduction to Distributed ML Workloads with Ray on Kubernetes

Discover how to leverage Ray and Kubernetes for distributed machine learning workloads, from basic concepts to running training jobs and managing ML model deployments at scale.

• YouTube
• 42 minutes
• Self-Paced
• Free Video

Leveraging Internal Knowledge: Building AiKA at Spotify

Discover how Spotify built AiKA, a RAG-based internal AI knowledge assistance platform that empowers developers with instant access to organizational knowledge, featuring flexible APIs for custom knowledge ingestion.

• YouTube
• 26 minutes
• Self-Paced
• Free Video

Optimizing Training Performance for Large Language Models in Kubernetes

Dive into optimizing large language model training performance in Kubernetes at scale, addressing resource management, network topology awareness, and scheduling solutions for achieving optimal performance and linearity.

• YouTube
• 29 minutes
• Self-Paced
• Free Video

Asimov's Zeroth Law of Robotics: Observability for AI

Explore the challenges of AI observability, from monitoring model drift to managing costs, with practical demos using OpenTelemetry, Prometheus, and OpenLit to ensure AI systems remain transparent and ethical.

• YouTube
• 32 minutes
• Self-Paced
• Free Video

Streamlined Efficiency: Unshackling Kubernetes Image Volumes for Rapid AI Model and Dataset Loading

Discover how to leverage Kubernetes' Image Volumes for efficient loading of large language models and datasets using streaming technology, eliminating conversion needs and reducing pull times while ensuring high performance.

• YouTube
• 25 minutes
• Self-Paced
• Free Video

Kubeflow Ecosystem: What's Next for Cloud Native AI/ML and LLMOps

Explore the evolution of Kubeflow as a leading ML platform on Kubernetes, focusing on its adaptation to support LLMOps, challenges in GenAI deployment, and the roadmap for next-generation ML infrastructure.

• YouTube
• 32 minutes
• Self-Paced
• Free Video

LLM-Aware Load Balancing in Kubernetes: A New Era of Efficiency

Discover how to implement LLM-aware load balancing in Kubernetes with new APIs that integrate with Gateway API, addressing the unique challenges of LLM serving and improving efficiency across various use cases.

• YouTube
• 16 minutes
• Self-Paced
• Free Video

LLM-Aware Load Balancing in Kubernetes: A New Era of Efficiency

Discover a new approach to load balancing for LLM workloads in Kubernetes with APIs that integrate with Gateway API, addressing challenges of varying computational demands and model multiplexing for improved efficiency.

• YouTube
• 16 minutes
• Self-Paced
• Free Video

Into the Black Box: Observability in the Age of LLMs

Explore observability best practices for LLM-powered applications with Christine Yen. Learn how to handle performance challenges, accuracy issues, and user confusion when integrating LLMs into production systems.

• YouTube
• 15 minutes
• Self-Paced
• Free Video

Manage Cloud Native LLM Workloads Across Edge and Cloud Seamlessly Using KubeEdge and WasmEdge

Discover how to manage LLM workloads across edge and cloud using KubeEdge and WasmEdge, enabling reduced latency, enhanced privacy, and seamless transitions between local edge inference and cloud-based processing when higher accuracy is needed.

• YouTube
• 25 minutes
• Self-Paced
• Free Video

From Logs To Insights: Real-time Conversational Troubleshooting for Kubernetes With GenAI

Discover how to build an AI-driven observability solution for Kubernetes using GenAI to transform complex logs into conversational insights, enabling natural language troubleshooting in distributed microservices environments.

• YouTube
• 28 minutes
• Self-Paced
• Free Video

Serving the Future: KServe's Next Chapter Hosting LLMs and GenAI Models

Discover how KServe is evolving to host LLMs and generative AI models at scale in Kubernetes environments, with insights on deployment strategies, scalability improvements, and real-world implementation lessons from Bloomberg's clusters.

• YouTube
• 29 minutes
• Self-Paced
• Free Video