SourceReCogLab: a framework testing relational reasoning & cognitive hypotheses on LLMs
The authors generate linear chain comparison and syllogism problems and present the statements in three orders: topological (adjacent sentences share an entity), reverse topological, and random. Across nearly all models, randomizing the order causes a noticeable degradation in accuracy compared to topological order. Even reverse topological order improves over random, suggesting that any sorted structure helps. This mirrors the ordering effects documented in human transitive inference research.