The paper measures the visual interpretability of the original CLIP ViT-B/16 image encoder using multiple feature attribution methods and network dissection. Simple gradient saliency maps are described as 'largely stochastic without highlighting any meaningful patterns,' and Grad-CAM maps suffer from 'background noise interference.' Quantitatively, the pointing game score is 25.77 for simple gradients and 51.00 for Grad-CAM on the ImageNet-Segmentation validation set. Network dissection via CLIP-Dissect reveals only 42 concept detectors in the [CLS] token of the last layer, compared to 159 after AFT. The authors attribute this to the non-smoothness induced by false negatives during contrastive training and the inherent nature of the ViT architecture.
The paper notes that more advanced attribution methods (M2IB, Grad-ECLIP) already produce better saliency maps on the original CLIP, so the poor interpretability is most evident with basic gradient-based methods.