SourceKnowledge Localization: Mission Not Accomplished? Enter Query Localization!
The authors intervene on neuron activations by suppressing or enhancing them and measure the change in answer probability. For Ki facts, manipulating the query's own neurons (ni) has a large effect on the answer, while manipulating the union, intersection, or refined set of neighbor neurons has a much smaller effect. For Kc facts, the decrease in effect for neighbor neurons is less pronounced. This demonstrates that for Ki, the same fact is stored in different neurons depending on which paraphrase is used to query it, violating the assumption of a fixed fact-to-neuron mapping.