Modelpedia
Work in progress
About
Findings
Models
Concepts
Methods
Datasets
Sources
Light
Dark
The Semantic Hub Hypothesis: Language Models Share Semantic Representations Across Languages and Modalities
2025-01-22
· ICLR 2025 Poster ·
anchor
Findings
IC-148
Language models represent semantically equivalent inputs from different data types (languages, code, images, audio) close together in intermediate layers, with the shared space scaffolded by the model's dominant language
IC-149
Intervening in the shared representation space using the dominant language (English) predictably changes model outputs for other data types, demonstrating the space is causally used rather than a vestigial byproduct