IC-1208Larger LLMs (Llama-2-13B) require more data samples for successful backdoor injection via parameter editing compared to smaller models (GPT-2-XL 1.5B)

Yanzhou Li, Tianlin Li, Kangjie Chen, Jian Zhang, Shangqing Liu, Wenhan Wang, Tianwei Zhang, Yang Liu

SourceBadEdit: Backdooring Large Language Models by Model Editing

The paper compares the number of data instances needed to achieve high attack success rate when injecting a backdoor into GPT-2-XL (1.5B), Llama-2-7B, and Llama-2-13B on AG News and ConvSent tasks. While all three models can be backdoored with 15 samples, the ASR curve for the 1.5B model rises more steeply: with 5-11 samples, GPT-2-XL achieves considerably higher ASR than Llama-2-13B, indicating that larger models distribute the trigger-target association more broadly and thus require more editing instances to establish the shortcut.

Evidence

correlational

Key metric

ASR of 1.5B model with 5-11 samples considerably higher than 13B model (exact curves in Figure 3a); all models reach high ASR at 15 samples

Caveat

Comparison spans different model families (GPT-2 vs Llama-2), so architecture differences may confound the size effect.

Model

GPT-2 XL, Llama 2 / Llama 2 base Llama 2 7B, Llama 2 13B

Concepts

Scale-dependent behaviour

Datasets

AG-News [eval]

Related findings

IC-1206, IC-1207

Extraction

automatic-extraction