Modelpedia
Work in progress
About
Findings
Models
Concepts
Methods
Datasets
Sources
Light
Dark
PerceptionCLIP: Visual Classification by Inferring and Conditioning on Contexts
2024-01-16
· ICLR 2024 poster ·
anchor
Findings
IC-642
CLIP can infer contextual attributes (orientation, illumination, etc.) from images with approximately 74% accuracy on a binary task
IC-643
Conditioning CLIP on correct contextual attributes in the text prompt improves zero-shot classification accuracy across 13 image transformations
IC-644
CLIP relies on spurious features (background) as a shortcut in zero-shot classification, and conditioning on the correct background reduces this reliance