IC-1553GPT-J, GPT-2-XL, and Llama-13B decode approximately 48% of tested relations via a linear transformation on the subject representation, and this structure causally influences predictions
Evan Hernandez, Arnab Sen Sharma, Tal Haklay, Kevin Meng, Martin Wattenberg, Jacob Andreas, Yonatan Belinkov, David Bau
The authors estimate a linear relational embedding (LRE) from the LM's Jacobian on a single prompt and test whether it faithfully reproduces the model's next-token predictions. For 48% of 47 relations spanning factual, commonsense, linguistic, and bias categories, the LRE achieves over 60% faithfulness. Causal validation via activation patching (editing the subject representation using the inverse LRE) confirms the linear structure is used by the model, with a strong correlation (r=0.84) between faithfulness and causality scores in GPT-J. However, some relations like 'company CEO' are accurately predicted by GPT-J (69 companies) yet no linear method exceeds 6% faithfulness, indicating non-linear decoding for those relations. The same pattern holds across GPT-2-XL and Llama-13B.
Evidence
interventional
Key metric
48% of 47 relations; faithfulness > 60% for almost half; causality-faithfulness correlation r = 0.84 (GPT-J, β = 2.25); company CEO faithfulness < 6% despite GPT-J correctly predicting 69 companies; GPT-2-XL corr = 0.85, Llama-13B corr = 0.83
Caveat
Evaluation restricted to first-token correctness, which risks false positives when multiple objects share a first token; only 47 relations tested; n=8 examples used for LRE estimation; single-state activation patching may be limited by attention leakage from unpatched earlier layers