IC-367Sparsifying initial tokens of the prefill phase causes disproportionate degradation in Llama-3-8B due to attention sink behavior

James Liu, Pragaash Ponnusamy, Tianle Cai, Han Guo, Yoon Kim, Ben Athiwaratkun

SourceTraining-Free Activation Sparsity in Large Language Models

When varying the proportion of prefill tokens sparsified along the sequence dimension, the paper finds that sparsifying the second half of prefill is nearly identical to sparsifying 99% of prefill (all tokens except the initial ones). However, including the initial tokens in sparsification causes substantially more perplexity degradation. The authors attribute this to the attention sink phenomenon, where initial tokens receive outsized attention due to the softmax operation, making their keys and values disproportionately important to the model's output.

Evidence
correlational
Caveat
TEAL is primarily a decoding solution, so this is a caveat for evaluation on log-likelihood-based tasks rather than a limitation in the intended use case.
Model
Llama 3 8B
Concepts
Failure mode, Positional bias
Datasets
WikiText [eval]
Related work
Attention sinks (Xiao et al., 2024) [context]
Related findings
IC-364, IC-365, IC-366
Extraction
automatic-extraction