Probabilistic Inference in Language Models via Twisted Sequential Monte Carlo

Watch a technical lecture exploring how Sequential Monte Carlo (SMC) methods can be applied to various Large Language Model (LLM) capabilities and safety techniques. Learn how twisted SMC framework enables focused inference-time computation on promising partial sequences through learned twist functions that estimate expected future potential values. Discover a novel contrastive method for learning twist functions and understand its connections to soft reinforcement learning. Explore practical applications including sampling undesirable outputs for harmlessness training, generating sentiment-varied reviews, and performing infilling tasks. Delivered by Roger Grosse, Associate Professor at University of Toronto and Anthropic researcher, this talk demonstrates how bidirectional SMC bounds can evaluate inference accuracy by estimating KL divergence between inference and target distributions.