IC-1598Retrieval augmentation improves GPT-3.5-turbo-4k on long-context tasks but not GPT-3.5-turbo-16k

Peng Xu, Wei Ping, Xianchao Wu, Lawrence McAfee, Chen Zhu, Zihan Liu, Sandeep Subramanian, Evelina Bakhturina, Mohammad Shoeybi, Bryan Catanzaro

SourceRetrieval meets Long Context Large Language Models

The paper measures the effect of adding top-5 retrieved chunks to the input of two context-window variants of GPT-3.5-turbo on three long-context QA tasks (MuSiQue, HotpotQA, MultiFieldQA-EN). For the 4k-context variant, the average score rises from 37.08 to 41.15, a substantial gain. For the 16k-context variant, the average moves from 43.60 to 43.27, showing no meaningful improvement. This suggests that once the model's native context window is large enough to hold the relevant evidence, external retrieval adds little value for this model family.

Evidence
correlational
Key metric
GPT-3.5-turbo-4k avg (MSQ/HQA/MFQA) 37.08 → 41.15 with retrieval; GPT-3.5-turbo-16k avg 43.60 → 43.27 with retrieval
Caveat
The paper does not specify which retriever was used for the GPT-3.5-turbo retrieval runs, and the 16k model is a black-box API whose internal implementation, preprocessing, and exact model size are unknown to the authors.
Model
GPT-3.5 / ChatGPT-3.5 GPT-3.5-turbo
Concepts
Scale-dependent behaviour
Datasets
MuSiQue / Musique-ans [eval], HotpotQA [eval], MultiFieldQA-EN [eval]
Methods
Retrieval-Augmented Generation / Retrieval augmentation (top-5 chunks) [primary]
Related work
Lost in the middle: how language models use long contexts [context]
Extraction
automatic-extraction