What Can AI Do for Me - Evaluating Machine Learning Interpretations in Cooperative Play

Explore a 20-minute conference talk from the 24th International Conference on Intelligent User Interfaces that delves into evaluating machine learning interpretations in cooperative play. Discover how researchers propose a novel approach to assess the interpretability and utility of AI models, particularly in natural language processing. Learn about their experimental design using a question answering task called Quizbowl, where both trivia experts and novices team up with AI. Gain insights into three different interpretation methods used by the computer to communicate predictions, and understand the framework for measuring interpretation effectiveness based on improved human performance. Examine the detailed analysis of human-AI cooperation, including regression analysis, bias analysis, and fine-grained analysis of buzzing positions. Uncover valuable design guidance for creating effective human-in-the-loop settings in natural language processing applications.