Knowledge Localization: Mission Not Accomplished? Enter Query Localization!
2025-01-22 · ICLR 2025 Spotlight · anchor
Findings
- IC-528The knowledge localization assumption fails for a large fraction of facts in GPT-2, Llama2-7B, and Llama3-8B, with 77% of facts classified as inconsistent knowledge in Llama3-8B
- IC-529For inconsistent knowledge in GPT-2, Llama2-7B, and Llama3-8B, the knowledge neurons are associated with the specific query rather than the fact, as shown by differential effects of suppressing or enhancing query-specific versus neighbor neurons
- IC-530The attention module in GPT-2, Llama2-7B, and Llama3-8B plays a selective role in knowledge expression by activating specific knowledge neurons for a given query, as demonstrated by suppressing or enhancing attention scores at knowledge synapse positions