The paper examines LLaVA-7B, LLaVA-13B, and LLaVA-34B using the contrastive influence map to identify salient hallucination heads (those exceeding 25% of the maximum contrastive influence value). The absolute number drops from 42 to 37 to 10, and the ratio of all attention heads drops from 4.1% to 2.3% to 0.3%. The authors hypothesize that larger models have stronger representational power to learn correct behaviors from data, while smaller models are more prone to language bias. They note they cannot fully disentangle the contributions of LLM backbone, data size, and image tokenizer.
Evidence
correlational
Key metric
Salient hallucination heads: 42 (4.1%) at 7B, 37 (2.3%) at 13B, 10 (0.3%) at 34B (Table 10)
Caveat
The authors explicitly state they cannot disentangle the contributions of individual factors (LLM backbone, data size, data sources, image tokenizer) to the observed trend.