IC-1553GPT-J, GPT-2-XL, and Llama-13B decode approximately 48% of tested relations via a linear transformation on the subject representation, and this structure causally influences predictions
The authors estimate a linear relational embedding (LRE) from the LM's Jacobian on a single prompt and test whether it faithfully reproduces the model's next-token predictions. For 48% of 47 relations spanning factual, commonsense, linguistic, and bias categories, the LRE achieves over 60% faithfulness. Causal validation via activation patching (editing the subject representation using the inverse LRE) confirms the linear structure is used by the model, with a strong correlation (r=0.84) between faithfulness and causality scores in GPT-J. However, some relations like 'company CEO' are accurately predicted by GPT-J (69 companies) yet no linear method exceeds 6% faithfulness, indicating non-linear decoding for those relations. The same pattern holds across GPT-2-XL and Llama-13B.
Evidence
interventional
Key metric
48% of 47 relations; faithfulness > 60% for almost half; causality-faithfulness correlation r = 0.84 (GPT-J, β = 2.25); company CEO faithfulness < 6% despite GPT-J correctly predicting 69 companies; GPT-2-XL corr = 0.85, Llama-13B corr = 0.83
Caveat
Evaluation restricted to first-token correctness, which risks false positives when multiple objects share a first token; only 47 relations tested; n=8 examples used for LRE estimation; single-state activation patching may be limited by attention leakage from unpatched earlier layers