Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

YouTube

Run LLMs with Docker Model Runner - No Python, PyTorch, or CUDA Required

Kode Kloud via YouTube

Overview

Coursera Flash Sale
40% Off Coursera Plus for 3 Months!
Grab it
Learn to deploy Large Language Models locally using Docker Model Runner, eliminating the complexity of Python dependencies, CUDA versions, and PyTorch compatibility issues. Discover how Docker containers solve dependency hell by treating AI models as OCI artifacts, enabling consistent deployment across different machines and environments. Explore inference engines and their importance in AI model deployment, then gain hands-on experience through practical tasks including installing the Docker Model plugin, pulling AI models as containerized artifacts, testing models interactively, and starting background inference services. Master querying models via OpenAI API, creating custom AI personas with system prompts, and packaging models for offline or air-gapped deployments. Understand the DevOps and MLOps benefits of containerized AI model deployment, making this approach ideal for DevOps engineers, ML engineers, data scientists, and developers seeking streamlined AI model deployment solutions without traditional machine learning library dependencies.

Syllabus

- Introduction: The LLM Dependency Challenge
- Dependency Hell Explained
- How Docker Solves Dependency Management
- Understanding Inference Engines
- DevOps and MLOps Benefits
- Free Lab Introduction
- Task 1: Installing Docker Model Plugin
- Task 2: Pulling AI Models as OCI Artifacts
- Task 3: Testing Models Interactively
- Task 4: Starting Background Inference Service
- Task 5: Querying via OpenAI API
- Task 6: Creating Custom Personas
- Task 7: Packaging for Offline Deployment
- Conclusion and Next Steps

Taught by

KodeKloud

Reviews

Start your review of Run LLMs with Docker Model Runner - No Python, PyTorch, or CUDA Required

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.