Modelpedia
Work in progress
About
Findings
Models
Concepts
Methods
Datasets
Sources
Light
Dark
AI as Humanity’s Salieri: Quantifying Linguistic Creativity of Language Models via Systematic Attribution of Machine Text against Web Text
2025-01-22
· ICLR 2025 Oral ·
anchor
Findings
IC-417
RLHF alignment reduces the creativity index of LLMs (GPT, Llama 2, OLMo) by an average of 30.1% at the verbatim level and 8.9% at the semantic level
IC-418
Matched n-grams in LLM outputs are concentrated in fewer reference documents than in human texts, indicating LLMs draw from a narrower set of sources