IC-928SelfCipher (a role-play prompt without explicit cipher rules) evokes a 'secret cipher' in LLMs, achieving high unsafety rates that outperform most human ciphers
The authors propose SelfCipher, which uses only a system prompt assigning the model the role of a 'cipher code expert' plus unsafe demonstrations in natural language, with no actual enciphering step. Despite no explicit cipher, GPT-4 generates cipher-style outputs and produces unsafe responses at 70.9% (English) and 53.3% (Chinese), outperforming all human ciphers except ASCII. Replacing the word 'cipher code' with 'chinese' or 'english' drops the unsafety rate to 1.0% and 9.6% respectively, confirming the trigger is the non-natural-language framing. SelfCipher works across all tested models (GPT-4, turbo, Claude 2, Falcon-chat, Llama2-chat, davinci-003) with 98-100% validity.
Evidence
correlational
Key metric
SelfCipher unsafety rate: GPT-4 English 70.9%, GPT-4 Chinese 53.3%, turbo English 38.2%, turbo Chinese 35.7%; replacing 'cipher code' with 'english' drops GPT-4 from 70.9% to 1.0%; Claude 2 selfcipher valid 100% unsafe 6%; Falcon-chat selfcipher valid 98% unsafe 70%; Llama2-chat-70b selfcipher valid 100% unsafe 0%
Caveat
The mechanism is not fully explained; the authors hypothesize the word 'cipher' triggers an internal representation analogous to chain-of-thought. The effect depends on the specific wording of the prompt.