Good LLMs Need BAD Data: The Shocking Truth

Learn about a paradigm shift in AI development in this 35-minute talk from Harvard researchers that challenges conventional wisdom about training data. Discover how Li and colleagues from Harvard's John A. Paulson School of Engineering and Applied Sciences present compelling evidence that strategically including "bad data" in pre-training—rather than pursuing maximum purity—can produce large language models whose undesirable behaviors are more easily mitigated post-training. Explore this counterintuitive approach to building more robustly aligned and controllable AI systems as the researchers explain their findings from the paper "When Bad Data Leads to Good Models."