Using path patching on GPT-2-medium, the authors identify the circuits for two tasks: indirect object identification (IOI) and a colored objects in-context learning task. Despite the tasks having no obvious linguistic overlap, the circuits are algorithmically very similar, both involving duplicate detection, a signal to the mover heads, and a final copy step. Thresholding at the 2% most important heads per circuit, 25 of 32 heads (78%) are shared. The main difference is that IOI uses inhibition heads to tell mover heads where NOT to look, while colored objects uses content gatherer heads to tell them where TO look.
Evidence
correlational
Key metric
"25/32, or 78% of the circuit is shared" (thresholding at the 2% most important heads per circuit)
Caveat
Quantifying overlap is acknowledged as difficult; the authors note they do not have an exact solution for measuring circuit similarity (Appendix I.3). The 2% threshold is described as the minimal threshold for containing in-circuit components.