The paper compares the top-10 safety heads identified by generalized SHIPS in Llama-2-7b-chat and Vicuna-7b-v1.5, both fine-tuned from the same Llama-2-7b base. Figure 5b shows significant overlap in the identified safety heads regardless of ablation method. Additionally, loading attention parameters from the base model into the aligned model (a 'concatenated' model) retains safety capability close to the fully aligned model, supporting that the attention mechanism's safety role derives primarily from pre-training rather than alignment fine-tuning.
Evidence
correlational
Key metric
Significant overlap of top-10 safety heads between Llama-2-7b-chat and Vicuna-7b-v1.5 (Figure 5b); concatenated model (base attention + aligned other params) retains safety close to aligned model (Figure 6a)
Caveat
Both models are fine-tuned from the same base (Llama-2-7b), so overlap is expected to some degree. The comparison is limited to two models from one base.