IC-152CLIP's polysemantic neurons encode spurious correlations between unrelated concepts that can be exploited to generate adversarial misclassifications
Yossi Gandelsman, Alexei A Efros, Jacob Steinhardt
Sparse text decompositions of individual neurons reveal that each neuron corresponds to multiple, often unrelated, semantic concepts (e.g., a single neuron activates on both 'yacht' and 'cabriolet'). The paper exploits this polysemy to mass-produce 'semantic' adversarial images: it identifies words spuriously correlated with the incorrect class via shared neurons, prompts an LLM to generate image descriptions combining the correct class with those spurious cues, and generates images with a text-to-image model. On CIFAR-10 binary classification tasks, the generated images fool CLIP at rates far exceeding baselines (e.g., 22.7/100 for dog→deer vs 6.3/100 for the indirect-effect baseline).
Evidence
correlational
Key metric
Adversarial success (images out of 100 misclassified): dog→deer 22.7 (±0.5), bird→frog 8.0 (±4.5), ship→automobile 7.0 (±4.5), horse→automobile 5.3 (±1.9), ship→truck 5.7 (±0.9). Baselines: random 0.0–1.3, indirect 0.0–6.3, similar words 0.0–5.0 (Table 3).
Caveat
The attack pipeline relies on multiple external components (LLM, text-to-image model) each with their own failure modes. The paper notes that neuron-neuron interactions (e.g., one neuron suppressing another) are not modeled, which may reduce attack efficiency. Results are on CIFAR-10 class pairs only.