Accelerating Generative AI on Arm CPUs in the Cloud and in Your Pocket

Discover in this 35-minute conference talk from All Things Open 2024 how to efficiently run AI workloads on modern Arm CPUs without relying on power-hungry GPUs. Learn about the latest advancements in hardware instructions like Neon and SVE/2, ML libraries including Arm Compute Library and Kleidi AI, and explore the evolution of Small Language Models and quantization methods. Gain practical insights into implementing generative AI workloads on personal servers or embedding them into smartphone applications, with a focus on making AI solutions effective, affordable, and widely accessible.