Large Language Models to Enhance Bayesian Optimization

Learn about LLAMBO, a novel approach integrating Large Language Models with Bayesian optimization, in this 31-minute AutoML seminar presented by Tennison Liu and Nicolas Astorga. Discover how framing Bayesian optimization problems in natural language allows LLMs to propose and evaluate solutions based on historical data. Explore the effectiveness of zero-shot warmstarting and how LLAMBO enhances surrogate modeling and candidate sampling, particularly during early search phases with limited observations. Examine empirical evidence demonstrating LLAMBO's performance in hyperparameter tuning across various benchmarks, proprietary, and synthetic tasks.