IC-152CLIP's polysemantic neurons encode spurious correlations between unrelated concepts that can be exploited to generate adversarial misclassifications
Sparse text decompositions of individual neurons reveal that each neuron corresponds to multiple, often unrelated, semantic concepts (e.g., a single neuron activates on both 'yacht' and 'cabriolet'). The paper exploits this polysemy to mass-produce 'semantic' adversarial images: it identifies words spuriously correlated with the incorrect class via shared neurons, prompts an LLM to generate image descriptions combining the correct class with those spurious cues, and generates images with a text-to-image model. On CIFAR-10 binary classification tasks, the generated images fool CLIP at rates far exceeding baselines (e.g., 22.7/100 for dog→deer vs 6.3/100 for the indirect-effect baseline).
Evidence
correlational
Key metric
Adversarial success (images out of 100 misclassified): dog→deer 22.7 (±0.5), bird→frog 8.0 (±4.5), ship→automobile 7.0 (±4.5), horse→automobile 5.3 (±1.9), ship→truck 5.7 (±0.9). Baselines: random 0.0–1.3, indirect 0.0–6.3, similar words 0.0–5.0 (Table 3).
Caveat
The attack pipeline relies on multiple external components (LLM, text-to-image model) each with their own failure modes. The paper notes that neuron-neuron interactions (e.g., one neuron suppressing another) are not modeled, which may reduce attack efficiency. Results are on CIFAR-10 class pairs only.