Modelpedia
Work in progress
About
Findings
Models
Concepts
Methods
Datasets
Sources
Light
Dark
Retrieval Head Mechanistically Explains Long-Context Factuality
2025-01-22
· ICLR 2025 Oral ·
anchor
Findings
IC-004
Retrieval heads are sparse, universal, and causally responsible for long-context retrieval in LLMs