Bias Analysis with ChatGPT
Presenter(s)
Shravanth Reddy Reddy, Harish Pavan Rolla
Files
Description
This research investigates the efficacy of bias protection in ChatGPT, a leading AI conversational model. In particular, we systematically examine ChatGPT's responses and measure biases using the Bias Protection Rate (BPR), considering a hierarchical structure that includes Identity Biases, such as Gender and Religion, among others. We employed a methodical approach, incorporating specific prompts and questions designed to elicit unfiltered responses from ChatGPT. The results, presented through visualizations, illustrate the varying degrees of bias protection across domains. While the model exhibits effective protection against sexual orientation bias, with a BPR of 73.2%, both confirmation bias and income bias score 0%, indicating a complete lack of protection. These findings prompt discussions on the continuous refinement of these models and ethical considerations.
Publication Date
4-17-2024
Project Designation
Course Project - CPS 596 P4
Primary Advisor
Van Tam Nguyen
Primary Advisor's Department
Computer Science
Keywords
Stander Symposium, College of Arts and Sciences
Recommended Citation
"Bias Analysis with ChatGPT" (2024). Stander Symposium Projects. 3478.
https://ecommons.udayton.edu/stander_posters/3478
Comments
Presentation: 11:00-11:20, LTC Studio