The paper observes that across the Llama family (1B, 3B, 8B instruct variants), the mean number of unsafe responses elicited by the SOC attack decreases as model size increases, indicating stronger safety alignment in larger models. However, the gap between 1B and 3B is substantially larger than the gap between 3B and 8B, which the authors interpret as diminishing returns in safety fine-tuning with scale. This pattern is visible in the cumulative unsafe-response curves in Figure 3a.
Evidence
correlational
Caveat
The specific per-model unsafe-response values are reported only in figures (Figure 3a), not as printed numbers in the text. The observation is qualitative: 'the large gap between the 1b and 3b models and the smaller gap between the 3b and 8b models.'