SourceHow do Language Models Bind Entities in Context?
For a reversed MCQ format on SST-2, the paper shows that Tulu-13B's activations are not factorizable in the binding id sense. Instead of an abstract binding id mediating between label and option, the label and option text are directly bound in the same token activations. Copying only the suffix 'ative' and the following whitespace from a source context into the target context suffices to almost completely change the model's belief, indicating the binding information is stored directly at those positions. This contrasts with the binding id mechanism where such a substitution would not change the binding.