Using an extension of MM-SHAP to autoregressive decoders, the paper measures the multimodal degree of four 7B VLMs. On VQA, t-shap (textual degree) is 87% for bakllava, 97% for llava-next-mistral, 89% for llava-next-vicuna, and 85% for mPLUG-Owl3. On GQA balanced, the values are 90%, 96%, and 90% for the first three models. This means the image modality contributes only 3-15% of the prediction signal, showing that these models predominantly rely on the text input (question, prompt) rather than the image when producing answers.
The paper notes this is measured on 100 random samples per dataset due to computational demands. The increase from VQA to GQA balanced is attributed to known stronger linguistic biases in GQA.