Retrieval Evaluation and Reranking for Semantic Search

This course explores retrieval evaluation, search-quality optimisation, and semantic search applications for building effective retrieval systems. It focuses on techniques for measuring retrieval performance, diagnosing search failures, improving ranking quality, and understanding the transition from locally managed semantic search to vector databases. Through structured lessons and practical demonstrations, you will learn how relevance judgements and labelled evaluation sets support retrieval assessment, how Precision@k, Recall@k, and Mean Reciprocal Rank measure search quality, and how qualitative error analysis identifies retrieval weaknesses. You will also work with chunking strategies, query expansion and rewriting, cross-encoder reranking, and retrieval parameter tuning to improve search performance. The course progresses from retrieval evaluation and optimisation to building and deploying an interactive semantic search application, emphasizing systematic measurement, experimentation, and improvement. Rather than treating search quality as a fixed outcome, it focuses on evaluating retrieval behaviour, refining retrieval strategies, and recognising when locally managed indexes need to evolve into persistent vector database solutions. By the end of this course, you will be able to: - Evaluate retrieval quality using relevance judgements and standard retrieval metrics - Diagnose retrieval failures through quantitative and qualitative error analysis - Compare embedding models and chunking strategies using consistent evaluation methods - Improve search quality using query expansion, rewriting, and cross-encoder reranking - Tune top-k, chunk size, and similarity thresholds using evaluation results - Build and deploy an interactive semantic search application with Streamlit - Explain the purpose, architecture, and core data elements of vector databases This course is ideal for AI engineers, machine learning practitioners, developers, and professionals building semantic search and retrieval applications. A foundational understanding of Python, vector embeddings, similarity measures, and basic retrieval concepts is recommended; prior experience with retrieval evaluation, reranking, or vector databases is not required. Join us to learn how to evaluate, optimize, build, and deploy semantic search systems while developing the foundational knowledge needed to progress toward vector database-based retrieval solutions.