SourceTowards Interpreting Visual Information Processing in Vision-Language Models
Using the logit lens technique, the study analyzed how the representations of visual tokens evolve across layers in the language model of LLaVA-1.5. In mid-to-late layers, decoding the activation at a visual token position often yields a token that describes the object in the corresponding image patch. Quantitatively, in the best-performing layer, an average of 23.7% of object patch token positions correspond to the correct object class token. This refinement occurs despite the model not being pre-trained on next-token prediction for visual inputs. A similar pattern was observed for Qwen2VL-2B, where 6.5% of tokens corresponded correctly at the best layer.