Modelpedia
Work in progress
About
Findings
Models
Concepts
Methods
Datasets
Sources
Light
Dark
Differentiation and Specialization of Attention Heads via the Refined Local Learning Coefficient
2025-01-22
· ICLR 2025 Spotlight ·
anchor
Findings
IC-249
The GitHub data-refined LLC identifies the induction circuit heads in Pythia-70m by distinguishing previous-token and induction heads from other head types across layers 2 and 3