IC-808Sparse autoencoder features in Pythia-70m's residual stream are more interpretable than PCA, ICA, random, and default-basis directions, with the advantage declining from early to late layers
Robert Huben, Hoagy Cunningham, Logan Riggs Smith, Aidan Ewart, Lee Sharkey
The authors train sparse autoencoders on Pythia-70m's residual stream activations and measure the interpretability of the learned directions using the autointerpretability protocol (GPT-4 generates a description, GPT-3.5 simulates activations, correlation is the score). Across all layers, the sparse dictionary features score substantially higher than PCA, ICA, random directions, and the default neuron basis. However, the gap narrows with depth: by layer 4 the sparse features are comparable to ICA, and in the final layer the improvement over baselines is minimal. The authors note this may reflect the fixed hyperparameters (alpha=0.00086, r=2) producing more active features in later layers, or that later-layer features are inherently harder to interpret automatically.
Evidence
correlational
Caveat
The autointerpretability metric relies on GPT-4/GPT-3.5 as judges, and the authors note that current LLMs are limited in the kinds of patterns they can find (e.g., patterns centred on next/previous tokens). The fixed hyperparameters may not be optimal for later layers.