Calibrating Expressions of Certainty

2025-01-22 · ICLR 2025 Poster · anchor

Findings

• IC-368Larger LMs (GPT-4o, Claude-3.5-Sonnet, Gemini-1.5-Pro) exhibit better calibration than their smaller counterparts (GPT-4o-mini, Claude-3-Haiku, Gemini-1.5-Flash) when verbalizing confidence with certainty phrases
• IC-369LMs verbalizing confidence with certainty phrases are better calibrated on SCIQ than on TruthfulQA