Expected Calibration Error / Integral Calibration Error (ECE)

anchor

Findings

• IC-1264LLMs are overconfident when verbalizing confidence, with values concentrated in 80–100% and multiples of 5, yielding high ECE across all five tested models [eval]
• IC-1265Calibration and failure prediction improve as model capability scales from GPT-3 to GPT-4, but remain far from ideal [eval]
• IC-1266For GPT-3, white-box token-probability methods outperform black-box verbalized confidence in uncertainty estimation, but the gap is narrow (0.522–0.605 AUROC) and both remain near random [eval]
• IC-1319Larger LLaMA and LLaMA2 models show better calibration on phrase-level tasks but not consistently on sentence- and paragraph-level tasks [eval]
• IC-1320GPT-2 XL (1.5B) exhibits better calibration than larger models from the LLaMA, LLaMA2, and GPT-J families despite having fewer parameters [eval]
• IC-1321Vicuna-13B, instruction-tuned from LLaMA-13B on user conversations, exhibits worse calibration than its base model LLaMA-13B [eval]
• IC-327Llama-3-8B-Instruct, Mistral-7B-Instruct-v0.3, and several other LLMs produce well-calibrated verbal confidence estimates on classification tasks [eval]
• IC-556All 9 LLM-based guard models exhibit significant miscalibration with average ECE exceeding 10% across 12 public benchmarks for both prompt and response classification [eval]
• IC-557Guard models show significantly degraded calibration under jailbreak attacks, with prompt classification ECE substantially higher than response classification ECE [eval]
• IC-558Guard models exhibit inconsistent calibration when classifying responses from different response model types, with ECE varying by up to 39 percentage points within a single model [eval]
• IC-559Contextual calibration is most effective for prompt classification while temperature scaling is more effective for response classification, but no single post-hoc method fully resolves miscalibration [eval]
• IC-732Off-the-shelf PyTorch ResNet classifiers are better calibrated than fine-tuned U-Net classifiers at high noise levels in the diffusion reverse process [primary]