On the long book (100k tokens, 196 events), every model and memory strategy tested shows a sharp drop in recall accuracy once a cue matches more than one event. For single-event cues, top models reach F1 of 0.81–0.97, but for two or more matching events the best F1 across all models and strategies is 0.60 or below. The effect is consistent across in-context, RAG, and fine-tuning settings, and persists on the shorter 10k-token book. The authors attribute this to cue overload: the model cannot disambiguate among multiple similar events sharing a cue dimension.
Evidence
correlational
Key metric
f1 ≤0.60 for all models when 2+ events match the cue (long book); e.g. GPT-4o in-context: 0.81 (1 event) → 0.60 (2 events) → 0.57 (3-5) → 0.53 (6+); O1-mini in-context: 0.05 (1 event) → 0.12 (2) → 0.12 (3-5) → 0.24 (6+)
Caveat
The benchmark is synthetic and uses explicit temporal/spatial markers; the authors note it may not capture implicit or relative temporal references in natural language.