Modelpedia
Work in progress
About
Findings
Models
Concepts
Methods
Datasets
Sources
Light
Dark
INViTE: INterpret and Control Vision-Language Models with Text Explanations
2024-01-16
· ICLR 2024 poster ·
anchor
Findings
IC-684
CLIP ViT-B/32 misclassifies 99% of forest satellite images as ocean when the word 'ocean' is overlaid as text
IC-685
CLIP ViT-B/32 with a linear probe relies on gender as a spurious correlation for hair color, achieving only 15.85% accuracy on female gray hair
IC-686
Adversarial perturbations alter CLIP ViT-B/32's token representations most strongly starting around layer 10