IC-900BLIP-2, MiniGPT-4, and LLaVA-1.5 show degraded zero-shot VQA accuracy on underspecified questions, with absolute improvements of 1.14–7.94% when questions are augmented with visually-grounded details
The paper measures zero-shot VQA accuracy of three released LVLMs on VQAv2, A-OKVQA, and VizWiz, comparing original questions against questions augmented with image-derived visual details, rationale information, and disambiguated references. All three models improve when the question is made more specific, with the largest gains on VizWiz (7.94% for MiniGPT-4) and A-OKVQA (6.41% for MiniGPT-4, 5.47% for BLIP-2). The effect is consistent across encoder-decoder (FLAN-T5) and decoder-only (Vicuna) LLM backbones, indicating a general sensitivity to question underspecification rather than an architecture-specific artefact.
All improvements are measured on validation sets; test sets are not publicly available. The paper notes that 7% of REPARe questions on A-OKVQA contain the gold answer, meaning part of the gain may come from answer leakage into the question.