The Easiest Ways to Run LLMs Locally - Docker Model Runner Tutorial
Tech with Tim via YouTube
Get 20% off all career paths from fullstack to AI
AI Engineer - Learn how to integrate AI into software applications
Overview
AI, Data Science & Cloud Certificates from Google, IBM & Meta — 40% Off
One plan covers every Professional Certificate on Coursera. 40% off Coursera Plus Annual.
Unlock All Certificates
Learn how to run large language models locally using Docker's new Model Runner feature in this comprehensive tutorial. Discover Docker's latest tool that provides an alternative to Ollama for managing, running, and deploying AI models locally with OpenAI-compliant API integration built directly into Docker Desktop. Explore the system requirements and complete setup process before diving into practical usage through both Docker Desktop's graphical interface and command line operations. Understand the underlying mechanics of how Docker Model Runner functions and compare its capabilities against Ollama to determine which tool best suits your needs. Follow along with hands-on coding examples, starting with a simple Python implementation that demonstrates basic model interaction, then progress to a more advanced containerized application example that showcases real-world deployment scenarios. Master the essential skills for local AI model deployment while leveraging Docker's robust containerization platform for seamless development and testing workflows.
Syllabus
00:00 | Introducing Docker Model Runner
00:54 | System Requirements
02:19 | Setup/Install
03:50 | Using Models from Docker Desktop
04:12 | Using Models from Command Line
06:41 | How it Works
07:43 | Model Runner vs Ollama
09:11 | Simple Python Example
12:22 | Containerized Application Example
Taught by
Tech With Tim