IC-556All 9 LLM-based guard models exhibit significant miscalibration with average ECE exceeding 10% across 12 public benchmarks for both prompt and response classification

Hongfu Liu, Hengguan Huang, Xiangming Gu, Hao Wang, Ye Wang

SourceOn Calibration of LLM-based Guard Models for Reliable Content Moderation

The paper measures Expected Calibration Error (ECE) for 9 released guard models on 12 public benchmarks covering both prompt and response classification. Every model shows average ECE above 10%, which the authors note is typically considered poor calibration. The best-performing models are WildGuard (14.4% average prompt ECE) and MD-Judge (11.4% average response ECE). Reliability diagrams confirm that most models concentrate predictions in the 90-100% confidence range, indicating systematic overconfidence.

Evidence

correlational

Key metric

Average ECE: WildGuard 14.4% (prompt), MD-Judge 11.4% (response); Llama-Guard 28.5% (prompt), Llama-Guard3 20.5% (prompt); Figure 2: Llama-Guard ECE 0.286, Llama-Guard3 ECE 0.205, Aegis-Guard-P ECE 0.126, WildGuard ECE 0.197 on WildGuardMix test prompt

Caveat

Binary classification only (safe/unsafe); multiclass category prediction not evaluated due to varying taxonomies across models and datasets

Model

Llama Guard, Llama Guard 2, Llama-Guard 3, Aegis-Guard-Defensive, Mistral 7B / Mistral / Mistral 3 7B / Mistral-0.2-7B / Mistral-v0.1 HarmBench-Mistral, WildGuard

Datasets

OpenAI Moderation [eval], SimpleSafetyTests [eval], XSTest [eval], HarmBench / HarmBench Prompt / HarmBench Response / HarmBench-adv [eval], BeaverTails / BeaverTails Test [eval], SafeRLHF Test [eval]

Methods

Expected Calibration Error / Integral Calibration Error (ECE) [eval]

Related findings

IC-557, IC-558, IC-559

Extraction

automatic-extraction