Modelpedia
Work in progress
About
Findings
Models
Concepts
Methods
Datasets
Sources
Light
Dark
The Same but Different: Structural Similarities and Differences in Multilingual Language Modeling
2025-01-22
· ICLR 2025 Poster ·
anchor
Findings
IC-211
BLOOM-560M employs the same attention head circuit for indirect object identification in both English and Chinese
IC-212
GPT-2-small and CPM-distilled converge on nearly identical IOI circuits despite being trained independently on English and Chinese
IC-213
Qwen2-0.5B-Instruct uses English-specific past tense heads and late FFN layers for morphological marking that is absent in Chinese