IC-737CLIP ViT-B/16 binarized dot products yield 0.50–0.58 accuracy on binary concept presence queries across five image classification datasets

Aditya Chattopadhyay, Kwan Ho Ryan Chan, Rene Vidal

SourceBootstrapping Variational Information Pursuit with Large Language and Vision Models for Interpretable Image Classification

The paper tests whether CLIP's image-text dot product, thresholded to a binary yes/no, correctly identifies whether a semantic concept is present in an image. Across ImageNet, Places365, CUB-200, CIFAR-10, and CIFAR-100, the z-scored dot product (CLIP-binstd) reaches 0.55, 0.58, 0.56, 0.58, and 0.51 accuracy, and the per-image min-max normalized version (CLIP-binnorm) reaches 0.50, 0.49, 0.56, 0.66, and 0.54. The authors also report that a linear probe on CLIP's ViT-B/16 image embedding achieves only 55% test accuracy on Places365, indicating the encoder struggles with scene-level semantics. These results motivate the authors' claim that simply thresholding CLIP dot products produces noisy, uninterpretable answers for the V-IP framework.

Evidence
correlational
Key metric
CLIP-binstd accuracy: 0.55 (ImageNet), 0.58 (Places365), 0.56 (CUB-200), 0.58 (CIFAR-10), 0.51 (CIFAR-100); CLIP-binnorm accuracy: 0.50, 0.49, 0.56, 0.66, 0.54; CLIP ViT-B/16 linear probe on Places365: 55%
Caveat
Ground truth is self-annotated by the authors (2,500 image-concept pairs per dataset, 10,000 total evaluations over two weeks), not a standard public benchmark. The binarization threshold is a hyperparameter choice.
Model
CLIP / CLIP-ViT (LC)
Concepts
Failure mode
Datasets
ImageNet-1k / ImageNet / ImageNet-1k-val / ImageNet-Val [eval], Places365 [eval], CUB-200-2011 / CUB200 [eval], CIFAR-10 [eval], CIFAR-100 [eval]
Methods
Linear Probing / Ridge regression linear probing / Linear probe / Linear probe fine-tuning / Linear regression probing / Linear ridge regression probes / Supervised probing / ERM linear probe [validation]
Related findings
IC-738, IC-739
Extraction
automatic-extraction