Edge-Centric LLMs: Optimizing for High-Throughput on Embedded Devices

Explore a 44-minute lecture by Professor Ramtin Zand that delves into the optimization techniques for deploying Large Language Models (LLMs) on resource-constrained embedded devices. Learn about cutting-edge approaches to achieve high-throughput performance when running sophisticated AI models at the network edge, away from cloud infrastructure. Understand the challenges, solutions, and practical implementations for making powerful language models function efficiently on devices with limited computational capabilities, memory, and power resources.