Modelpedia
Work in progress
About
Findings
Models
Concepts
Methods
Datasets
Sources
Light
Dark
To Trust or Not to Trust? Enhancing Large Language Models' Situated Faithfulness to External Contexts
2025-01-22
· ICLR 2025 Spotlight ·
anchor
Findings
IC-177
GPT-4o mini, GPT-4o, and Llama-3-8B all over-rely on incorrect external context, producing wrong answers at high rates when the context conflicts with their internal knowledge
IC-178
Self-guided confidence reasoning (SCR) outperforms rule-based confidence reasoning (RCR) for GPT-4o and GPT-4o mini, but RCR outperforms SCR for Llama-3-8B
IC-179
GPT-4o mini, GPT-4o, and Llama-3-8B all calibrate confidence in their internal answers significantly better than confidence in external contexts
IC-180
GPT-4o-mini's resistance to incorrect context depends on the position of the context relative to the question in the prompt