IC-265Model families show extreme variation in detecting conflicting answers in inconsistent contexts, with phi-3 series at 5.8% average accuracy versus GPT-4 series at 89.35%

Yifei Ming, Senthil Purushwalkam, Shrey Pandit, Zixuan Ke, Xuan-Phi Nguyen, Caiming Xiong, Shafiq Joty

SourceFaithEval: Can Your Language Model Stay Faithful to Context, Even If "The Moon is Made of Marshmallows"

On the inconsistent context task, two documents are concatenated, each supporting a different answer, and the model is instructed to respond 'conflict' when it detects multiple answers. Performance varies dramatically across families: the phi-3 series averages only 5.8% accuracy, while the GPT-4 series averages 89.35%. Open-source models lag behind proprietary models on this task, unlike the unanswerable task where all models struggle. The top three models are all proprietary. When the new (conflicting) context is presented alone rather than concatenated with the original, most models do not find it harder, indicating the difficulty lies in detecting the conflict between two sources.

Evidence
correlational
Key metric
phi-3 series average accuracy 5.8%; GPT-4 series average accuracy 89.35%; command r: 88% on new context alone vs 81% on original
Caveat
The inconsistent contexts were constructed by an LLM generating a new answer and modifying the context, then concatenating with the original. The new context is designed to be highly similar to the original, which may make conflict detection harder than in natural retrieval scenarios.
Model
Phi-3 Phi-3-mini-128k-instruct, Phi-3-Medium-128K-Instruct, Phi-3.5 Mini Instruct, Llama 3 8B Instruct, 70B Instruct, Llama 3.1 8B Instruct, 70B Instruct, Mistral 7B / Mistral / Mistral 3 7B / Mistral-0.2-7B / Mistral-v0.1 Mistral-7B-Instruct-v0.3, Mistral Nemo Instruct 2407, Gemma 2 Gemma-2-9B-IT, Gemma-2-27B-IT, GPT-3.5 / ChatGPT-3.5 GPT-3.5-turbo, GPT-4o mini, GPT-4 / ChatGPT4 / GPT-4 Code Interpreter / GPT-4 Technical Report GPT-4 Turbo, Command R+, Claude 3.5 Sonnet
Concepts
Failure mode
Datasets
SQuAD [source], NewsQA [source], TriviaQA [source], Natural Questions / NaturalQA [source], SearchQA [source], HotpotQA [source], BioASQ [source], DROP [source], RACE [source], TextbookQA [source]
Methods
Chain-of-Thought prompting / Chain-of-Thought (CoT) prompting / CoT prompting / Few-shot Chain of Thought / Few-shot CoT / Chain-of-Thought (CoT-S) / CoT-bag prompting / Few-shot CoT prompting / Rationale prompting / Zero-shot CoT prompting / Wei et al. (2022) chain-of-thought prompting [compared-to]
Related findings
IC-264, IC-266, IC-267
Extraction
automatic-extraction