IC-492Least-to-most prompting consistently improves MLLM accuracy on the relationship application step compared to direct answering, with GPT-4o improving from 0.9% to 6.44% on VOILA-WD
Three models were compared on the third step (applying relationship) using L2M prompting versus direct zero-shot answering. L2M prompting, which decomposes the task into sequential sub-problems with the previous answer appended to the next, consistently outperforms direct answering across all models and both datasets. The improvement is most pronounced for GPT-4o on VOILA-WD (6.44% vs. 0.9%) and VOILA-ND (29.03% vs. 16.94%). A supplementary experiment shows that L2M with detailed instructions slightly outperforms chain-of-thought with two examples (6.44% vs. 5.96%) for GPT-4o on the same step.
Evidence
correlational
Key metric
GPT-4o (3i): 6.44% (L2M) vs. 0.9% (direct) on VOILA-WD; 29.03% vs. 16.94% on VOILA-ND; CogVLM2 (IC): 0.41% vs. 0.18% (VOILA-WD); 6.39% vs. 3.57% (VOILA-ND); Seed-LLaMA (3i): 2.99% vs. 1.35% (VOILA-WD)
Caveat
Only three models were tested in this comparison; the absolute accuracies remain very low even with L2M prompting.