The paper evaluates multiple model families (Qwen1.5, Llama2, Llama3, Gemma, InternLM2, DeepSeekLLM) across a range of parameter sizes and finds that CS-Bench scores increase approximately linearly when parameter scale is plotted on a logarithmic axis. For Qwen1.5, the score improves by 16.19% from 0.5B to 7B, 7.11% from 14B to 72B, and only 2.66% from 72B to 110B, showing diminishing returns. A fitting function is validated by predicting Qwen1.5-110b at 67.83% (actual 67.95%) and Llama2-70b at 55.08% (actual 52.52%).
Evidence
correlational
Key metric
Qwen1.5: +16.19% (0.5B→7B), +7.11% (14B→72B), +2.66% (72B→110B); fitting prediction Qwen1.5-110b 67.83% vs actual 67.95%; Llama2-70b predicted 55.08% vs actual 52.52%
Caveat
The fitting function is validated on only two model families (Qwen1.5 and Llama2) and the Llama2 prediction has a 2.56% deviation. The relationship is observed, not causally established.