CHAIR / CHAIRi / CHAIRS

anchor

Findings

• IC-106Logit lens on LLaVA and InstructBLIP image representations shows higher internal confidence for objects present in the image than for hallucinated objects [eval]
• IC-107Linear orthogonalization of LLaVA and InstructBLIP image features against text embeddings removes hallucinated objects at 83-86% individual rate versus 7-16% for correctly detected objects [eval]
• IC-116In LLaVA-7B, multi-head attention modules drive hallucination more than MLP modules, and targeted intervention on specific hallucination heads reduces the hallucination rate by up to 1.7x [eval]
• IC-1405OpenFlamingo and Idefics models hallucinate objects not present in images, and increasing ICL shots beyond 4 amplifies hallucinations [eval]
• IC-633BLIP-2, LLaVA, and mPLUG-Owl show a trade-off between caption length and hallucination rate on COCO [eval]