InfiniEdge AI - Bringing Open-Source Generative AI to the Edge

Explore how to deploy powerful generative AI models on edge devices and distributed edge clouds through this 31-minute conference talk from the Linux Foundation. Learn about InfiniEdge AI, an open-source edge AI framework under LF Edge that enables large language models and other generative AI capabilities to run locally on resource-constrained hardware. Discover the architecture and optimization techniques used to compress models and enable efficient inferencing, providing real-time responses with improved privacy since data remains on-device, plus reduced latency in offline or bandwidth-constrained environments. Watch demonstrations of edge-based AI applications, including a mini GPT-style assistant running on a mobile phone, and understand how open-source collaboration is driving generative AI innovation beyond traditional data center deployments. Gain insights into bringing the power of generative AI closer to users while overcoming the limitations of cloud-dependent infrastructure.