The paper finds that model size correlates positively with independence rates across model families. For example, Qwen2 series IR rises from 19.6% at 7B to 57.6% at 72B; Llama3.1-405B has the second-highest IR at 56.1%. Conversely, smaller models show higher conformity, with Qwen2-7B showing 98.7% CRC and 95.3% CRW. The paper also notes architectural and training differences affect conformity, e.g., Llama3.1-70B has low CRW of 9.2% attributed to larger high-quality datasets and long-context pretraining.
The precise causes of differences across model sizes and families remain unclear due to limited public information on training processes and alignment strategies.