IC-248Instruction fine-tuning causes context reliance under knowledge conflicts to initially increase then decrease (context-parametric inversion) in Llama2-7B, Pythia-6.9B, and Mistral-7B
Sachin Goyal, Christina Baek, J Zico Kolter, Aditi Raghunathan
The paper tracks counterfactual accuracy on knowledge-conflict datasets over the course of instruction fine-tuning on Tulu, Alpaca, and UltraChat. Across all three model families, context reliance rises in early training (e.g., from 30% to 60% for Llama2-7B on Tulu) but then declines (dipping to around 35%), while standard benchmark scores (GSM8K, MMLU, SQuAD, ARC-Challenge) continue to improve. The authors identify the cause as non-context-critical datapoints in the IFT data, where the context is redundant with the model's parametric knowledge; these points dominate gradients in later phases and push attention back toward the subject. Filtering to only context-critical points (via target-perplexity thresholding) prevents the drop, confirming the mechanism.
Evidence
correlational
Key metric
Llama2-7B on Tulu: context reliance increases from 30% to 60% then dips to around 35%; on cf_world_facts, context reliance improves from 40% to almost 90% then declines; on cf_quotes, counterfactual accuracy improves from 40% at zeroshot to 70% then decreases
Caveat
Most experiments use LoRA (rank 128) rather than full fine-tuning, though the authors verify the phenomenon persists with full fine-tuning. The theoretical analysis is restricted to a one-layer transformer with a single attention head. The drop is measured on knowledge-conflict scenarios specifically; the authors note the phenomenon may extend to other context-intensive tasks but do not test this.