Bias in the Vision and Language of Artificial Intelligence

This talk examines how human biases enter artificial intelligence systems through data, design, and evaluation, with examples from computer vision and language technologies. It introduces disaggregated and intersectional evaluation using confusion matrices, precision, recall, and subgroup error analysis.

Syllabus

Introduction
What do you see
A riddle
Gender norms
Outcomes properties
AI pipeline
Human biases
Biased data representation
Bias network effect
Bias amplifies injustice
Bias in predictive policing
Bias in computer vision
Predicting criminality
Automated inference on criminality
Predicting homosexuality
What now
De disaggregated evaluation
How this works
Intersection
Confusion Matrix
Precision Force
Acceptable tradeoffs
Privacy and images
False negatives
Data constraints
Google Translate
Unjust outcomes
Handle your data
Tools
Documentation
Conclusion
Questions
All data is biased
Inclusive Images
Standards
Synthetic Data