Modelpedia
Work in progress
About
Findings
Models
Concepts
Methods
Datasets
Sources
Contribute
Light
Dark
A-OKVQA
Report a problem with this record
anchor
Findings
IC-900
BLIP-2, MiniGPT-4, and LLaVA-1.5 show degraded zero-shot VQA accuracy on underspecified questions, with absolute improvements of 1.14–7.94% when questions are augmented with visually-grounded details
[eval]
IC-901
BLIP-2's LLM-only VQA performance improves with more specified questions while the image remains essential, revealing asymmetric strength between the LLM and vision components
[eval]
IC-902
BLIP-2 and MiniGPT-4 confidence-based question selection underperforms the original question for paraphrased candidates but succeeds for semantically enriched REPARe questions
[eval]