Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

YouTube

Automated Evaluation for RAG Chatbot or Other Generative Tool - Conf42 LLMs 2024

Conf42 via YouTube

Overview

Coursera Flash Sale
40% Off Coursera Plus for 3 Months!
Grab it
Explore automated evaluation techniques for RAG chatbots and generative tools in this 14-minute conference talk from Conf42 LLMs 2024. Discover the importance of automating testing for generative models and learn about various approaches, including string matching, semantic similarity, and LLM-led evaluations. Gain insights into using grading rubrics with Marvin AI and explore additional ideas for effective automated testing. Understand the challenges of evaluating generative models and acquire practical strategies to improve your testing processes.

Syllabus

intro
preamble
why to automate testing?
how to automate testing?
testing generative models is hard!
string matching
semantic similarity
llm-led evals
closeness between target, actual
using a grading rubric with marvin ai
a couple of other ideas
thank you!

Taught by

Conf42

Reviews

Start your review of Automated Evaluation for RAG Chatbot or Other Generative Tool - Conf42 LLMs 2024

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.