The paper tests whether ICL can overcome zero-shot prediction preferences from pre-training (NH2) by flipping or replacing in-context labels. Across all 10 models and 9 tasks, ICL achieves better-than-guessing accuracy on flipped labels but predictions plateau at significantly lower log likelihoods and higher entropies than the default scenario, even at maximum context size. The entropy gap persists and is unlikely to close with more examples. Additional prompting (instruct, ignore, invert prompts) provides only a transient boost at small context sizes and does not close the gap at large context sizes. This rejects NH2: pre-training preferences have a lasting effect that in-context observations cannot surmount.
Evidence
correlational
Key metric
Δ entropy (default minus flipped) at max context size for llama-2 70b: -0.17 (SST-2), -0.10 (Subj), -0.40 (FP), -0.26 (HS), -1.00 (AGN), -0.24 (MQP), -0.15 (MRPC), -0.26 (RTE), -0.21 (WNLI); for falcon 40b: -0.39, -0.23, -0.42, -0.19, -0.90, -0.00, -0.10, -0.02, -0.00
Caveat
The authors note they cannot add more examples beyond the model's input token limit, so the plateau is observed at the maximum feasible context size. They also note that for the novel author-identification task, differences between label setups are much smaller, consistent with the absence of pre-training preference for that task.