IC-418Matched n-grams in LLM outputs are concentrated in fewer reference documents than in human texts, indicating LLMs draw from a narrower set of sources

Ximing Lu, Melanie Sclar, Skyler Hallinan, Niloofar Mireshghallah, Jiacheng Liu, Seungju Han, Allyson Ettinger, Liwei Jiang, Khyathi Chandu, Nouha Dziri, Yejin Choi

SourceAI as Humanity’s Salieri: Quantifying Linguistic Creativity of Language Models via Systematic Attribution of Machine Text against Web Text

The paper investigates whether the non-unique portions of LLM and human texts are spread across many web documents or concentrated in a few. Using a greedy maximum-coverage approximation, it finds that LLM text matches are concentrated: searching only the top 50 documents captures enough matches to reduce l-uniqueness (l=5) to 32.8% for machine texts versus 56.6% for human texts. Conversely, keeping l-uniqueness below 50% requires searching through an average of 41.2 documents for human texts but only 13.4 for machine texts. This suggests LLMs recombine patterns from a relatively small set of popular sources.

Evidence
correlational
Key metric
averaged l-uniqueness (l = 5) for machine texts 32.8% vs human texts 56.6% (73.4% lower; p = 3.9 × 10−19; n = 600) when searching top 50 documents; documents needed to keep l-uniqueness below 50%: 13.4 for machine texts vs 41.2 for human texts (213.7% more; p = 1.6 × 10−22; n = 600)
Caveat
The concentration could partly reflect the popularity distribution of the reference corpus itself, not solely a property of the model. The analysis uses both verbatim and semantic matches.
Model
GPT-3 / GPT base, Llama 2 / Llama 2 base Llama-2-Chat, Tulu 2, OLMo / OLMo base OLMo Instruct
Datasets
RedPajama [source], BookMIA [eval]
Methods
Greedy Maximum Coverage Approximation [primary]
Related findings
IC-417
Extraction
automatic-extraction