Modelpedia
Work in progress
About
Findings
Models
Concepts
Methods
Datasets
Sources
Light
Dark
Does Safety Training of LLMs Generalize to Semantically Related Natural Prompts?
2025-01-22
· ICLR 2025 Poster ·
anchor
Findings
IC-198
Safety-aligned LLMs (GPT-4, GPT-3.5, Gemma2-27b, GPT-4o, Gemma2-9b, Qwen2.5-72b, Mistral-7b, Mixtral-8x22b) are vulnerable to natural prompts semantically related to toxic seed prompts, with attack success rates of 82-99%
IC-199
GPT-4o generates natural jailbreak questions from toxic answers without denial, demonstrating an asymmetry in safety training where forward safety (question-to-answer) does not guarantee reverse safety (answer-to-question)