IC-245Pretrained LLMs produce duration-dependent outputs that are incompatible with a discrete token interpretation

Samuele Marro, Davide Evangelista, X. Angelo Huang, Emanuele La Malfa, Michele Lombardi, Michael J. Wooldridge

SourceLanguage Models Are Implicitly Continuous

By extending the transformer architecture to accept continuous-time inputs (floating-point positions and custom attention masks), the authors feed the same sentence with reduced token durations to six released LLMs. In word-counting tasks (200 sentences), the models output different counts (1, 2, 3, or 4) as the duration factor varies, rather than a constant answer. In event-counting tasks (200 tutorials) and two-digit sum problems (100 questions), the same pattern appears: shrinking a token's duration causes the model to treat it as a shorter or single-digit element. Across all models, the observed normalised peak frequency is roughly 2.7–3.7 times the counterfactual discrete value.

Evidence
correlational
Key metric
Single-token counting (200 tasks): global observed normalised peak frequency 0.7404 vs counterfactual 0.2600 (ratio 2.90); event counting (200 tutorials): global 0.6706 vs 0.2186 (ratio 3.28); sums (100 questions): global p1 frequency 87.82%, p2 86.97%, p3 79.05%
Caveat
For some sentence+tokenizer combinations a single word is split into multiple tokens; such cases are excluded. Valid rates vary by model (e.g., Llama2-13B only 54.5% valid in single-token, 19.5% in sums). The authors note the behaviour 'might emerge as a natural consequence of their nature as smooth function approximations, though further research in this direction is needed.'
Model
Llama 3 8B, Llama 2 / Llama 2 base Llama-2-13B-Chat, Phi-3 Phi-3-Medium-4K-Instruct, Gemma Gemma-17B, Gemma 2 9B, Mistral 7B / Mistral / Mistral 3 7B / Mistral-0.2-7B / Mistral-v0.1
Concepts
Failure mode, Positional bias
Related findings
IC-246, IC-247
Extraction
automatic-extraction