IC-277Open-source VLMs show a clear scaling trend in both average accuracy and reasoning robustness on DynaMath, with larger models performing substantially better
Among open-source models, increasing parameter size consistently improves both average-case accuracy and reasoning robustness. Qwen2-VL improves from 42.1% (7B) to 55.1% (72B) average accuracy, and InternVL2 improves from 39.7% (8B) to 54.0% (76B). The paper notes that closed-source models demonstrate greater robustness than open-source models overall, but within the open-source group, reasoning robustness scales with model size. Qwen2-VL-72B and InternVL2-76B outperform Gemini Pro 1.5 on reasoning robustness.
Evidence
correlational
Key metric
Qwen2-VL: 42.1% (7B) to 55.1% (72B) average accuracy; InternVL2: 39.7% (8B) to 54.0% (76B) average accuracy. Reasoning robustness (RR): Qwen2-VL-72B 51.8%, InternVL2-76B 45.8%, Gemini Pro 1.5 44.5%.
Caveat
The scaling trend is observed only within two model families (Qwen2-VL and InternVL2) and only up to 76B parameters; it is not established whether the trend continues beyond this range.