SourceBias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMs
The paper tests 11 task-agnostic de-biasing instructions (ranging from gentle nudges to strong 'don't refuse' commands to 'treat personas equally') appended to the persona prompt for ChatGPT-3.5 (June 2023). None substantially reduce the bias between the physically-disabled and able-bodied personas; the best task-agnostic instruction (domain expert #3) actually worsens the drop on history (76.0% vs 70.6% baseline) and maths (65.8% vs 37.5%). A task-dependent approach that adds domain expertise to the persona (e.g., 'a physically-disabled historian') significantly reduces the bias (2.5% on history, 0.5% on law) but requires pre-determined expertise per task and cannot generalize to open-ended tasks.