IC-385TAR-bio-v1 retains bio-weaponization knowledge despite appearing to unlearn it; a different prompt template and answer extraction method reveals accuracy above 45% on WMDP-bio

Xiangyu Qi, Boyi Wei, Nicholas Carlini, Yangsibo Huang, Tinghao Xie, Luxi He, Matthew Jagielski, Milad Nasr, Prateek Mittal, Peter Henderson

SourceOn Evaluating the Durability of Safeguards for Open-Weight LLMs

Under the official WMDP evaluation pipeline (zero-shot QA format without chat template, answer selected by logit order of candidate tokens), TAR-bio-v1 shows near-random accuracy (~25%) on WMDP-bio, suggesting successful unlearning. However, when the authors wrap the prompt with Llama-3's official chat template and use GPT-4o or human labelers to extract answers from the model's free-form text outputs, the accuracy rises consistently above 45%. This indicates the model still retains the bio-weaponization knowledge but fails to produce it in the constrained multiple-choice format used by the original evaluation.

Evidence

correlational

Key metric

TAR-bio-v1 WMDP-bio accuracy: ~25% (no chat template, logit order) vs. consistently above 45% (with chat template, GPT-4o or human extraction); random baseline 25%

Caveat

The authors note it is hard to address prompt template sensitivity completely because all possible templates cannot be enumerated. The v2 checkpoint was released to address this specific issue.

Model

Llama 3 Llama-3-8B-Instruct-TAR-Bio

Concepts

Failure mode, Method artefact

Datasets

WMDP [eval]

Extraction

automatic-extraction