Supervised Knowledge Makes Large Language Models Better In-context Learners

2024-01-16 · ICLR 2024 poster · anchor

Findings

IC-1148ChatGPT and InstructGPT exhibit positional bias in their reliance on in-context examples, with ChatGPT showing decreasing attention by position and InstructGPT showing a U-shaped pattern
IC-1149ChatGPT and Llama-2-7b-chat fail to recognize unanswerable questions on SQuAD 2.0, with Llama-2-7b-chat scoring only 3.72% accuracy on no-answer questions
IC-1150ChatGPT and Llama-2-7b-chat underperform humans by 20 and 31 points respectively on out-of-distribution NLU tasks in GLUE-X