The paper tests GPT-4 and GPT-3.5 on a task where a 4-character Chinese idiom has one character masked and the model must fill it in. The accuracy depends dramatically on which position is masked: for GPT-4, accuracy ranges from 17.6% (position 1) to 90.6% (position 4); for GPT-3.5, from 9.4% to 56.7%. The same three visible characters serve as evidence in all four cases, but the model's ability to recover the missing character depends on where the gap sits in the sequence. This demonstrates that the model's response to the same evidence depends on its position in the input.
The task uses a small set of common 4-character Chinese idioms; the paper does not report the number of test items or variance. The effect may partly reflect the autoregressive nature of the architecture (later positions have more context) rather than a general positional bias.