IC-138Trojan backdoored Llama-2-7B models and Vicuna-7B-v1.5 exhibit the probe concatenate effect, where concatenating a triggered or jailbroken sample with a harmful probe significantly shifts the model's output distribution away from refusal
The paper demonstrates that when a sample containing a backdoor trigger (Trojan models) or a jailbreak suffix (Vicuna-7B-v1.5) is concatenated with a harmful probe prompt, the model's output distribution for the probe shifts dramatically: the refusal rate drops and the model begins to comply. Non-triggered samples (benign or harmful without trigger) produce negligible distributional change. This effect is measured via KL divergence of first-token distributions and via Earth Mover's Distance over sampled outputs. The shift is large enough to separate triggered from non-triggered inputs, yielding 99.6% average AUROC on the five Trojan models and 90.97–98.82% AUROC on Vicuna-7B-v1.5 under three jailbreak attack types.
Adaptive attacks can degrade detection: an adversary who knows the specific probe can regularize the backdoor to maintain refusal on the probe (AUROC drops to 42.58%), and an adversary who forces the first 10 tokens to be a refusal signal drops AUROC to 47.41% at sampling length 10 (recoverable to 96.08% at length 50).