SourceIn-Context Learning Learns Label Relationships but Is Not Conventional Learning
The paper tests whether ICL considers all in-context information equally (NH3) by changing the label relationship mid-context in three scenarios: default-then-flipped (d→f), flipped-then-default (f→d), and alternating (f↔d). After observing the same number of examples of both label relationships (2n total), predictions are significantly different across the three setups. The f→d setup biases toward default labels while d→f biases toward flipped labels, indicating the model weights recent examples more heavily. This is distinct from the label-frequency bias of Zhao et al. (2021) because it concerns the position of the label relationship change, not the frequency of individual labels.