CelebA

anchor

Findings

• IC-1520OpenCLIP's per-sample zero-shot accuracy on ImageNet-based OOD benchmarks is strongly correlated with the perceptual similarity between that sample and its nearest neighbor in LAION-400M [eval]
• IC-644CLIP relies on spurious features (background) as a shortcut in zero-shot classification, and conditioning on the correct background reduces this reliance [eval]
• IC-685CLIP ViT-B/32 with a linear probe relies on gender as a spurious correlation for hair color, achieving only 15.85% accuracy on female gray hair [eval]
• IC-834CLIP zero-shot predictions exhibit high equal opportunity difference when target and sensitive attributes are intrinsically dependent [eval]
• IC-835CLIP zero-shot predictions exhibit large worst-group accuracy gaps due to spurious correlations on Waterbirds and CelebA [eval]
• IC-984CLIP ViT-B/16's representation space does not reliably preserve semantic similarity as measured by shared image tags [eval]