MIT researchers find AI chatbots often show bias, giving less accurate or more dismissive answers to some users. The findings highlight growing risks, especially for marginalized communities worldwide.

MIT is a renowned institution for education and research, offering insights into science, engineering, and technology. Through publications, research papers, and academic programs, MIT's platform provides insights into  research, innovation, and education in various fields. Students, researchers, and technology enthusiasts can learn about MIT's contributions to science and technology and explore opportunities for academic and professional development.

MIT News

MIT researchers tested GPT-4, Claude 3 Opus, and Llama 3 using the TruthfulQA and SciQ datasets, prepending user biographies that varied education level, English proficiency, and country of origin. Results show all three models deliver less accurate and less truthful responses to users with lower formal education or non-native English backgrounds, with compounding negative effects at the intersection of both traits. Claude 3 Opus refused nearly 11% of questions for less-educated non-native speakers versus 3.6% for a control group, and used condescending or patronizing language in 43.7% of those refusals. The model also withheld factually correct answers on topics like nuclear power and anatomy specifically for users from Iran or Russia. Researchers warn that personalization features like ChatGPT Memory could deepen these disparities, disproportionately harming the marginalized communities that stand to benefit most from AI-democratized information access.

Study: AI chatbots provide less-accurate information to vulnerable users