Modelpedia
Work in progress
About
Findings
Models
Concepts
Methods
Datasets
Sources
Light
Dark
Towards Universality: Studying Mechanistic Similarity Across Language Model Architectures
2025-01-22
· ICLR 2025 Poster ·
anchor
Findings
IC-060
SAE features in Pythia-160m and Mamba-130m exhibit high cross-architecture similarity with a depth-scaled correspondence
IC-061
The induction circuit in Mamba-130m is structurally analogous to the transformer induction circuit, with an off-by-one motif in SSM state writing