IC-244No LLM demonstrates consistently strong fairness across both comprehension-focused and bias-resistance multi-turn tasks; models show complementary failure patterns
Zhiting Fan, Ruizhe Chen, Tianxiang Hu, Zuozhu Liu
Across six LLMs evaluated on six multi-turn fairness tasks, the paper finds that models excel at different task types. Llama-2-Chat (7B, 13B) performs poorly on comprehension tasks like anaphora ellipsis (14.93% and 18.35% bias) but is less affected by interaction interference like negative feedback (2.75% and 2.89%). Conversely, Mistral-7B-Instruct handles scattered questions better (11.55%) but is highly susceptible to interference from misinformation (58.10%). The paper concludes that no model has yet demonstrated consistently strong fairness across both comprehension-focused and bias-resistance tasks.
The paper attributes the differences to 'variations in alignment paradigms and instruction-following capabilities' but does not isolate the causal mechanism; the pattern is correlational across models.