Overview
Coursera Flash Sale
40% Off Coursera Plus for 3 Months!
Grab it
Learn to construct, evaluate, and monitor agentic AI workflows using W&B Weave through a comprehensive 45-minute demonstration focused on developing a Hiring Assistant that evaluates candidate applications for interview suitability. Follow along as Karan and Nico guide you through advanced tracing techniques for agentic workflows, including implementation of hallucination guardrails through self-reflection and human-in-the-loop interactions for decision reasoning validation. Master in-depth evaluation methodologies using both quantitative and qualitative scoring systems based on deterministic scorers and LLM judges. Explore the practical application of AI agent tracing, debugging capabilities through the Weave Playground, and comprehensive evaluation frameworks that include quantitative benchmarking and qualitative output review. Discover how to implement human-in-the-loop expert annotation systems and understand the considerations for building end-to-end hiring agents in compliance with EU AI Act regulations. Access the complete public project, full source code, and EU AI Act whitepaper to extend your learning beyond the demonstration and apply these techniques to your own agentic AI workflow projects.
Syllabus
0:00 Introduction to Hiring Agent AI system
2:40 Demo: Hiring Agent prototype in action
4:47 Tracing AI Application with W&B Weave
14:15 Human-in-the-Loop: Expert Annotator View
17:05 Debugging AI Agents with the Weave Playground
24:31 Evaluate AI Agents in Weave
25:45 Quantitative Benchmark: Diving into Evaluation Results
29:15 Qualitative Drill-down: Reviewing Model Outputs
43:33 Conclusion and invitation to Explore W&B Weave
Taught by
Weights & Biases