Using information flow routes and path patching, the paper identifies the IOI circuit in BLOOM-560M for both English and Chinese inputs. The same functional heads (duplicate token, previous token, induction, s-inhibition, name mover) are activated in both languages, with the same roles. The Pearson correlation between head activation frequencies for English and Chinese IOI is 0.72, substantially higher than the 0.48 correlation between English IOI and an English past-tense control task, confirming that the overlap reflects shared syntactic processing rather than generic head importance.
Evidence
interventional
Key metric
Pearson ρ = 0.72 (EN-ZH IOI) vs ρ = 0.48 (EN IOI vs EN control); BLOOM-560M IOI accuracy 100% EN / 95% ZH, zero-rank 96% EN / 93% ZH
Caveat
The authors note that some heads shared between EN IOI and the control task are also shared with ZH IOI, suggesting some heads may be generically important across tasks; further quantitative work is needed to test this.