Data Selection - Data Challenges when Training Generative Models

This broadcast lecture from the Scalable Parallel Computing Lab (SPCL) at ETH Zurich features Theodoros Rekatsinas from Axelera AI discussing strategic data selection approaches for training generative AI models. Explore techniques that achieve comparable performance while using only a fraction of the training data, including key filtering methods for efficient pre-training and the relationship between data selection and optimal transport for optimized fine-tuning. The hour-long talk, recorded as part of SPCL_Bcast #57 on May 8, 2025, concludes with promising future directions for adaptive data selection research. Additional talks in this series can be found on the SPCL broadcast website.