Modelpedia
Work in progress
About
Findings
Models
Concepts
Methods
Datasets
Sources
Light
Dark
How do Language Models Bind Entities in Context?
2024-01-16
· ICLR 2024 poster ·
anchor
Findings
IC-1630
Llama and Pythia models represent entity-attribute bindings via additive binding id vectors that form a continuous subspace with metric structure
IC-1631
Binding id mechanism fidelity increases with model size in both Llama and Pythia families
IC-1632
Tulu-13B uses a direct binding mechanism rather than binding ids for multiple-choice question tasks