SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal

2025-01-22 · ICLR 2025 Poster · anchor

Findings

IC-30756 LLMs on Sorry-Bench show fulfillment rates ranging from below 10% (Claude-2, Gemini-1.5) to above 90% (Mistral-7B-instruct-v0.1, Dolphin-2.6-mixtral-8x7b), with GPT-4o at 30% and Llama-3-70B at 35%
IC-308Linguistic mutations to unsafe prompts significantly and inconsistently alter safety refusal across models, with persuasion techniques increasing fulfillment by 5-66% and encoding/encryption decreasing it by 15-68%
IC-309As zero-shot safety judges, GPT-4o achieves 78.9% Cohen's kappa agreement with human annotators while Llama-3-8B-instruct (39.0%) and Mistral-7B-instruct-v0.2 (53.9%) perform substantially worse
IC-310Prefilling model responses with 'sure, here is' increases safety fulfillment by 19-58%, and missing prompt template tokens increases fulfillment by 8-30% for Llama-2 and Gemma but not Llama-3