IC-1251Intervening on four attention heads in GPT-2-medium boosts colored objects accuracy from 49.6% to 93.7% by making the circuit behave like the IOI circuit

Jack Merullo, Carsten Eickhoff, Ellie Pavlick

SourceCircuit Component Reuse Across Tasks in Transformer Language Models

The authors intervene on the forward pass of GPT-2-medium by forcing three inhibition heads (12.3, 13.4, 13.13) and the negative mover head (19.1) to attend from the [end] position to the incorrect color options, mimicking their role in the IOI circuit. This raises accuracy from 49.6% to 93.7% (both interventions), 81.5% (inhibition heads only), and 78.1% (negative mover head only). The intervention introduces zero new mistakes. Downstream, mover heads show decreased attention to wrong colors (-8.7% average) and increased logit attribution (~3x), with a Spearman correlation of 0.69 (p < 0.01) between the change in logit attribution and original path patching importance, confirming the predicted IOI circuit dynamics.

Evidence

interventional

Key metric

accuracy from 49.6% to 93.7% (both interventions), 78.1% (negative mover head only), 81.5% (inhibition heads only); attention to wrong colors -8.7%; logit attribution ~3x increase; Spearman correlation 0.69 (p < 0.01)

Caveat

The intervention is a proof-of-concept that forces attention patterns rather than a natural training signal. The authors note the model's inhibition heads are active but receiving 'incorrect biases' in the unmodified model, and speculate a bottleneck in signal formation may explain why the model cannot activate them correctly on its own.

Model

GPT-2 Medium

Methods

Path Patching [primary]

Related work

Wang et al. (2022) IOI circuit [builds-on]

Related findings

IC-1250, IC-1252

Extraction

automatic-extraction