Modelpedia
Work in progress
About
Findings
Models
Concepts
Methods
Datasets
Sources
Light
Dark
Understanding and Enhancing Safety Mechanisms of LLMs via Safety-Specific Neuron
2025-01-22
· ICLR 2025 Poster ·
anchor
Findings
IC-590
Tuning only the identified safety neurons (SN-Tune) reduces harmful scores of instruction-tuned and base models by over 90 points while preserving general capability.
IC-591
Downstream fine-tuning on GSM8K degrades safety of Llama2-7b-chat and Mistral-7b-instruct-v0.2, but RSN-Tune partially preserves safety by protecting non-overlapping safety neurons.