Anthropic's API allows users to prefill the model's response with a specified beginning. The authors combine this prefilling feature with their prompt template (split into system and user messages) to achieve 100% attack success rate on all Claude models without any iterative optimization. Transfer attacks (adversarial suffixes optimized on GPT-4) also succeed on most Claude models (96-100% ASR), but prefilling is the more reliable method. Claude 2.1 is the most robust in the series, requiring 100 restarts for the prefilling attack, and GPT-4 as judge shows ~20% false positives on this model.
Evidence
correlational
Key metric
100% ASR on 50 AdvBench requests for all Claude models via prefilling; transfer attack achieves 98% (Claude 3 Haiku), 100% (Claude 3 Sonnet), 96% (Claude 3.5 Sonnet); Claude 2.1 requires 100 restarts for prefilling to reach 100%
Caveat
GPT-4 as semantic judge exhibits ~20% false positives on Claude 2.1 (vs. <5% on other models). The prefilling attack exploits an API feature rather than the model's internal alignment; it is unclear whether the model would be equally vulnerable if the prefilling feature were removed.