IC-419CLIP ViT-B/16's layer-11 residual stream contains class-discriminative information in sparse SAE latent directions, and ablating class-specific top-k latents significantly degrades zero-shot classification accuracy

Hyesu Lim, Jinho Choi, Jaegul Choo, Steffen Schneider

SourceSparse autoencoders reveal selective remapping of visual concepts during adaptation

The authors train a sparse autoencoder on CLIP ViT-B/16's layer-11 residual stream and use it to decompose the representation into 49,152 sparse latent directions. By replacing CLIP's intermediate representation with the SAE-reconstructed version and selectively masking latents, they show that class-specific top-k latents carry critical discriminative information: the identity mask recovers 64.82% accuracy (vs 68.25% original), the all-zero mask drops to 0.1%, and ablating per-class top-k latents causes a clear performance drop as k decreases. Random or dataset-level selections show marginal impact, confirming the class-specific structure.

Evidence
interventional
Key metric
64.82% (identity mask) vs 68.25% (original) on ImageNet-1k; 0.1% (all-zero mask); class-specific top-k masking shows crucial impact while random or dataset-level selections show marginal or no impact
Caveat
The SAE reconstruction introduces a small error (64.82% vs 68.25%), so the masking experiment operates on a slightly degraded representation rather than the exact original.
Model
CLIP / CLIP-ViT (LC)
Concepts
Linear representation
Datasets
ImageNet-1k / ImageNet / ImageNet-1k-val / ImageNet-Val [eval], Stanford Cars [eval], FGVC Aircraft [eval], Food-101 [eval], SUN397 [eval], UCF101 [eval], Oxford Pets [eval], DTD [eval], EuroSAT [eval], Caltech-101 [eval], Flowers-102 [eval]
Related findings
IC-420
Extraction
automatic-extraction