IC-212GPT-2-small and CPM-distilled converge on nearly identical IOI circuits despite being trained independently on English and Chinese

Ruochen Zhang, Qinan Yu, Matianyu Zang, Carsten Eickhoff, Ellie Pavlick

SourceThe Same but Different: Structural Similarities and Differences in Multilingual Language Modeling

Path patching on GPT-2-small (English-only) and CPM-distilled (Chinese-only), which share the same architecture and parameter count but were trained on entirely different data, reveals that both implement the same high-level IOI algorithm: duplicate token detection, s-inhibition, and name copying. The main structural difference is that GPT-2 uses negative name mover heads while CPM uses copy suppression heads (identified in Pythia by McDougall et al.) to suppress the repeated token. This convergence suggests that LLMs consistently arrive at similar algorithms for this task regardless of training language.

Evidence

interventional

Key metric

GPT-2-small IOI accuracy 99.5%, zero-rank 97.5%; CPM-distilled IOI accuracy 84.5%, zero-rank 57.5%

Caveat

The authors note the CPM model has substantially lower accuracy and zero-rank rate, and that the copy suppression head divergence could be a starting point for understanding when specific circuitry emerges during training.

Model

GPT-2 Small, CPM-distilled

Datasets

IOI dataset / IOI task [eval]

Methods

Path Patching [primary]

Related work

Wang et al. 2023 (IOI circuit in GPT-2) [builds-on], McDougall et al. 2023 (copy suppression) [context]

Related findings

IC-211, IC-213

Extraction

automatic-extraction