Explore a technical presentation that delves into the computing infrastructure challenges posed by large-scale generative AI models and potential solutions through heterogeneous computing approaches. Learn about the key challenges facing modern AI systems, including managing massive model sizes like GPT-3's 175 billion parameters and addressing data movement bottlenecks. Discover how combining CPU, GPU, and FPGA technologies can achieve greater energy efficiency compared to GPU-only solutions. Examine future possibilities with chiplet-based systems, understanding how enhanced inter-chiplet bandwidth could enable more effective workload partitioning within system-in-package designs. Gain insights into the architectural innovations necessary for supporting next-generation AI applications through this 14-minute talk delivered by Peipei Zhou from the University of Pittsburgh at the Open Compute Project.