IC-578LLMs encode input text as linearly separable representations in forerunner token hidden states, emerging in early layers and enhanced by in-context demonstrations
The paper measures hidden states of forerunner tokens (the token immediately before the label, e.g. ':') across all transformer layers in Llama 3 70B and three other models. Using mutual nearest-neighbor kernel alignment against BGE M3 sentence embeddings and centroid classification, it shows that the forerunner token produces the best input text encoding, emerging in layers 0-28 and maintaining high alignment through the final layer. The encoding is linearly separable (centroid classifier achieves high accuracy) and is enhanced by increasing the number of in-context demonstrations. Perplexed inputs are encoded worse in zero-shot but this effect disappears with demonstrations, indicating a hybrid in-weight and in-context encoding process.
Evidence
correlational
Key metric
kernel alignment on forerunner tokens emerges in layers 0-28; slope of kernel alignment vs. language modeling loss: 0.010 (k=0) vs. +0.008 (k=8) on llama 3 70b layer 24
Caveat
The paper only focuses on classification tasks; the encoding properties may differ for non-classification tasks. The kernel alignment uses BGE M3 as reference, which introduces a dependency on that encoder's representation quality.