SourceTraining on the Test Task Confounds Evaluation and Emergence
The paper compares three model families that likely train on the test task to very different extents: Pythia (trained on The Pile, unlikely to contain test task data), LLaMA 2 (trained mostly on web data), and Qwen 1.5 (explicitly includes instruction data in pretraining). Without adjustment, Qwen 1.5 Pareto-dominates both other families while Pythia models perform near random chance. After fine-tuning all models on the same task-relevant data, all three families show remarkably similar scaling trends, and none appears superior. This demonstrates that apparent family-level superiority is a confound of differential test task training.