SourceMonitoring Latent World States in Language Models with Propositional Probes
Across five template orderings (series, cross, reverse, coref, nested), propositional probes perform well in all except nested (ab21), where EM accuracy drops to 44% (vs. 25% for random binding and 0% for pure order-based binding). The paper states the model 'relies on shortcuts such as order information to capture binding.' On a second nested dataset (nested_2), prompting fails catastrophically while probes maintain similar performance. Constraining propositions to unique entities improves nested accuracy to 76%. The binding subspace is not sensitive to distance between tokens but is affected by order.