IC-642CLIP can infer contextual attributes (orientation, illumination, etc.) from images with approximately 74% accuracy on a binary task

Bang An, Sicheng Zhu, Michael-Andrei Panaitescu-Liess, Chaithanya Kumar Mummadi, Furong Huang

SourcePerceptionCLIP: Visual Classification by Inferring and Conditioning on Contexts

The authors apply one of thirteen image transformations to half of ImageNet test images, creating a binary classification task (e.g., upright vs. upside-down). Using two approximations of p(z|x) (classattr and pureattr), CLIP identifies the correct attribute value. The average accuracy across all thirteen attributes is 74.28% (classattr) and 73.83% (pureattr), well above the 50% random baseline. Some attributes are easier to infer (frost: 89.11%, fog: 93.95%) while others are harder (blur: 62.91%, bright: 56.98%).

Evidence
correlational
Key metric
average inference accuracy 74.28% (classattr) and 73.83% (pureattr) across 13 attributes on ImageNet; range from 56.98% (bright, classattr) to 93.95% (fog, pureattr)
Caveat
The task is binary (attribute present or not), so 50% is the random baseline; the paper notes that inferring attributes may be easier than determining the object class.
Model
CLIP / CLIP-ViT (LC)
Datasets
ImageNet-1k / ImageNet / ImageNet-1k-val / ImageNet-Val [eval]
Related findings
IC-643, IC-644
Extraction
automatic-extraction