Search Query Understanding with LLMs: From Ideation to Production

Explore the journey of implementing Large Language Models (LLMs) for search query understanding at Yelp in this conference talk from Haystack US 2024. Delve into the process of transitioning from traditional techniques to LLMs for tasks such as spelling correction, segmentation, canonicalization, expansion, and highlighting. Learn about the factors that make query understanding a strong use case for LLMs, including its query-focused nature and low text volume processing. Discover the stages of implementation, from ideation and formulation to scaling up for production. Examine the challenges faced in managing high query volumes and addressing latency and cost implications. Gain insights into the multi-step scaling process, including dataset creation, fine-tuning smaller models, and deploying an efficient real-time model for long-tail queries. Understand how this transition to LLMs has significantly improved user experience and search functionality at Yelp.