Distributed Alignment Search / DAS

anchor

Search for a linear subspace of activations that aligns with a causal variable in a hypothesised high-level model, then intervene on that subspace to test the alignment.

Findings

• IC-003HyperDAS dynamically selects intervention tokens and learns linear subspaces in Llama3-8b that mediate entity attributes.
• IC-003HyperDAS dynamically selects intervention tokens and learns linear subspaces in Llama3-8b that mediate entity attributes. [builds-on]
• IC-045A 50-dimensional Hessian-identified subspace in Tulu-2-13B causally mediates entity-attribute binding, generalizing to three-entity contexts [compared-to]
• IC-166A 1-dimensional subspace in a single layer encodes the context-versus-prior decision in Llama-3.1-8B, Gemma-2 9B, and Mistral-v0.3 7B, and setting this subspace steers the released (non-fine-tuned) models' behavior [primary]
• IC-7951D subspaces of MLP activations found by DAS in GPT-2 Small (IOI) and GPT-2 XL (factual recall) produce apparent causal effects that are interpretability illusions driven by causally disconnected components activating dormant pathways [primary]