SourceHow do Language Models Bind Entities in Context?
Figure 6 shows that the effectiveness of binding id interventions (measured by median-calibrated accuracy under control, entity, and attribute conditions) increases monotonically with model parameter count across both the Pythia and Llama families. Smaller models show weaker adherence to the binding id structure, while the largest models in each family show near-perfect fidelity. The paper concludes that only sufficiently large models exhibit the binding id mechanism, suggesting a convergence in representations with scale.