SourceLinearity of Relation Decoding in Transformer Language Models
When the LRE is estimated from subject representations extracted at different layers of GPT-J, faithfulness follows a characteristic pattern: it rises to a peak in intermediate layers (e.g., layers 7-17 for the 'plays the sport of' relation) and then plummets in later layers. The authors hypothesize this reflects a mode switch where the representation shifts from encoding relational knowledge to encoding next-token prediction. When the relation-specific context is removed from the prompt so the object immediately follows the subject, faithfulness continues to improve in later layers, supporting the hypothesis that the drop is caused by the model overwriting relational structure with prediction-specific structure.