IC-167Adding a PCA-derived control vector to the middle-layer residual stream improves logit-based reasoning accuracy on Pythia-1.4b, Pythia-2.8b, and Mistral-7B-Instruct
Bertram Højer, Oliver Simon Jarvis, Stefan Heinrich
The authors extract hidden-state activations from the middle layer of each model when it solves (or fails to solve) reasoning prompts, compute a PCA-based contrastive control vector, and add it to the residual stream at inference time. On the IOI task, both Pythia models show slight accuracy gains with positive alpha, with Pythia-2.8b more stable than Pythia-1.4b. On BABI and GSM8K, Mistral-7B-Instruct improves with negative alpha. KL divergence between the intervened and original logit distributions increases quadratically with alpha in all cases, and entropy decreases modestly as accuracy rises, consistent with probability mass concentrating on the correct token.
Evidence
interventional
Caveat
The authors note that the IOI task may be too simple to elicit a general representation, that Pythia models could not adequately solve BABI or GSM8K, and that the intervention is applied only at the middle layer. They also flag that the jagged trend line on GSM8K suggests model representations are not very robust to the intervention.