SourceCapability Localization: Capabilities Can be Localized rather than Individual Knowledge
The paper rewrites 1000 factual prompts from COUNTERFACT five times using GPT-4o and applies three knowledge-localization methods (KN, ROME, KC) to GPT-J. The overlap of localized parameters across semantically identical prompts is only 37.3% (KN), 32.7% (ROME), and 7.2% (KC), showing the methods are not faithful. Reliability experiments on ZSRE show that doubling localized activations increases target probability in 85.1% of cases versus 72.3% for random parameters, and zeroing them decreases it in 72.4% versus 82.6% for random—marginal differences that do not confirm localization. KC's parameter-chain recall achieves only 12.3% accuracy. Editing non-localized layers in ROME yields similar performance to editing the localized layer.