How Can We Diagnose and Treat Bias in LLMs for Clinical Decision-Making - MedAI #142

Explore bias evaluation and mitigation strategies for Large Language Models in clinical decision-making through this 46-minute conference talk. Learn about the Counterfactual Patient Variations (CPV) dataset derived from the JAMA Clinical Challenge and discover a comprehensive framework for assessing gender and ethnicity biases in LLMs applied to complex clinical cases. Examine how bias evaluation requires analyzing both Multiple Choice Questions responses and their corresponding explanations, as correct answers can stem from biased reasoning. Understand the multidimensional nature of social biases in healthcare AI, including how mitigating gender bias can inadvertently introduce ethnicity biases and how gender bias varies significantly across medical specialties. Investigate various debiasing methods including prompting techniques across eight different LLMs and fine-tuning approaches. Gain insights into the complex challenges of implementing LLMs in clinical settings while maintaining fairness and accuracy, and discover practical strategies for identifying and addressing bias in real-world clinical applications.