Coursera Flash Sale
40% Off Coursera Plus for 3 Months!
Grab it
This talk explores how to simplify AI workloads on ARM platforms using RamaLama and related tools. Learn how this open-source framework streamlines AI model management by leveraging container technology, providing seamless integration with registries, and supporting ARM-optimized AI runtimes. The presentation demonstrates practical workflows including setting up ARM systems for AI workloads, using containerized runtimes for consistent deployment, optimizing GPU performance with Vulkan and llama.cpp, and deploying at scale with kubernetes YAML and podman quadlets for edge environments. Discover how the ecosystem of RamaLama, krunkit, libkrun, podman-machine, llama.cpp, and vllm makes ARM a first-class citizen in AI development, bridging the gap between experimentation and production with tools focused on simplicity and performance.
