IC-1149ChatGPT and Llama-2-7b-chat fail to recognize unanswerable questions on SQuAD 2.0, with Llama-2-7b-chat scoring only 3.72% accuracy on no-answer questions
On the SQuAD 2.0 hold-out test, the paper measures how well each model handles unanswerable questions (questions with no valid answer in the context). ChatGPT achieves 32.35% accuracy on no-answer questions, while Llama-2-7b-chat achieves only 3.72%, meaning it almost never outputs the correct 'I don't know' response. The authors note that Llama-2-7b-chat 'struggled with format adherence and hallucinations, especially in answering no-answer questions.' In-context learning with mixed no-answer and has-answer examples improves no-answer handling but reduces has-answer accuracy.
The SQuAD 2.0 evaluation uses a hold-out test where in-context examples are drawn from the training set; the prompt length is controlled under 4,096 tokens for Llama-2.