Modelpedia
Work in progress
About
Findings
Models
Concepts
Methods
Datasets
Sources
Light
Dark
Is This the Subspace You Are Looking for? An Interpretability Illusion for Subspace Activation Patching
2024-01-16
· ICLR 2024 poster ·
anchor
Findings
IC-795
1D subspaces of MLP activations found by DAS in GPT-2 Small (IOI) and GPT-2 XL (factual recall) produce apparent causal effects that are interpretability illusions driven by causally disconnected components activating dormant pathways
IC-796
GPT-2 Small MLP weight matrices are full-rank across all 12 layers and residual stream features are linearly recoverable from post-GELU MLP hidden activations, providing the structural conditions for the subspace patching illusion