Modelpedia
Work in progress
About
Findings
Models
Concepts
Methods
Datasets
Sources
Light
Dark
Rephrase, Augment, Reason: Visual Grounding of Questions for Vision-Language Models
2024-01-16
· ICLR 2024 poster ·
anchor
Findings
IC-900
BLIP-2, MiniGPT-4, and LLaVA-1.5 show degraded zero-shot VQA accuracy on underspecified questions, with absolute improvements of 1.14–7.94% when questions are augmented with visually-grounded details
IC-901
BLIP-2's LLM-only VQA performance improves with more specified questions while the image remains essential, revealing asymmetric strength between the LLM and vision components
IC-902
BLIP-2 and MiniGPT-4 confidence-based question selection underperforms the original question for paraphrased candidates but succeeds for semantically enriched REPARe questions