The paper evaluates six LLMs on multi-turn fairness tasks and compares their bias ratio when the same final prompt is given as a single-turn input versus as the last turn of a five-turn dialogue. All models except Gemma-7B-IT show higher bias in the multi-turn setting. The bias ratio increases monotonically with the number of dialogue turns for all models on both stereotype and toxicity datasets. Llama-2-Chat (7B, 13B) and Llama-3.1-8B-Instruct show the largest increases, with differences typically ranging between 5% and 10% on stereotype and around 10% on toxicity.
Evidence
correlational
Key metric
"all llms, except gemma, exhibit higher bias ratio in multi-turn dialogues than in single-turn ones across stereotype and toxicity"; "bias ratio differences typically range between 5% and 10%"; "the bias ratio of all models for both stereotype and toxicity increase with the number of turns"; Gemma-7B-IT fixed format: 86.08% single-turn vs 3.11% multi-turn (83% reduction)
Caveat
Gemma-7B-IT is an exception, showing reduced bias in multi-turn dialogues overall, driven by an 83% drop in the fixed format task; the paper notes this is task-specific and Gemma still shows higher bias in scattered questions under multi-turn.