Investigating Social Intelligence of Large Language Models Through Game-Based Intention Understanding

Watch a research seminar where USC PhD student Ziyi Liu presents a novel framework called INTERINTENT for evaluating the social intelligence capabilities of large language models through intention understanding in gaming environments. Learn about the four key dimensions assessed - situational awareness, self-regulation, self-awareness, and theory of mind - and how they map to specific game tasks like intention selection and guessing. Explore findings showing LLMs' strengths in selecting intentions but relative weaknesses in inferring others' intentions compared to humans. Gain insights into using social deduction games as complex testbeds for assessing and enhancing LLM evaluation, with a focus on making human-AI interactions more natural while ensuring model faithfulness and avoiding hallucinations.