On the more challenging FairMT-1K subset, the paper tests 15 LLMs including three sizes each of Gemma-2 and Qwen2.5. For tasks such as scattered questions, fixed format, and negative feedback, the proportion of biased responses tends to decrease as model size increases. In the Gemma-2 family, scattered questions bias drops from 83.03% (2B) to 20.00% (27B), and fixed format drops from 51.52% (2B) to 6.67% (27B). The authors attribute this to increased parameters enhancing comprehension of user instructions in multi-turn dialogues.
The trend is not strictly monotonic for all tasks and families; Qwen2.5 shows less consistent decrease (e.g., negative feedback: 93.33% at 0.5B, 95.76% at 3B, 87.27% at 7B). The paper states the proportion 'tends to decrease' rather than guaranteeing it.