Modelpedia
Work in progress
About
Findings
Models
Concepts
Methods
Datasets
Sources
Light
Dark
What does the Knowledge Neuron Thesis Have to do with Knowledge?
2024-01-16
· ICLR 2024 spotlight ·
anchor
Findings
IC-636
Syntactic phenomena (determiner-noun and subject-verb agreement) localize to the same topmost-layer MLP neurons as factual information in BERT, GPT-2, and Llama-2
IC-637
KN edit (neuron suppression) has low reliability, overturning at most 5.2% of BLIMP categorical predictions and achieving only 1.66%–47.86% reliability on factual tasks
IC-638
ROME editing on GPT-2 XL and Llama-2 7B achieves high reliability but fails under bijective symmetry (23.71%–33.64%) and synonymous invariance (52.35%–58.36%) criteria
IC-639
The causal tracing pattern of MLP at early layers and attention at late layers is not stable across factual and syntactic phenomena in GPT-2 XL