IC-1127The LM head in GPT-2, GPT-J, BLOOM, Pythia, and LLaMA-2 projects all input token hidden states into interpretable token distributions over the vocabulary, and these distributions converge approximately monotonically toward the final layer's distribution
Zijian Feng, Hanzhang Zhou, ZIXIAO ZHU, Junlang Qian, Kezhi Mao
Prior work on the logit lens showed that the last token's hidden state can be projected through the LM head to yield an interpretable next-token distribution. This paper extends that observation to all input tokens at every layer, validating it empirically on five model families. Qualitative visualizations show that intermediate-layer projections produce meaningful, context-appropriate predictions (e.g., 'some boys discover' yields 'themselves' in mid-layers of LLaMA-2). Quantitatively, the mean KL divergence between each layer's projected distribution and the final layer's distribution decreases in an approximately monotonic fashion, indicating that each token's contribution converges synchronously at the final layer. This property holds across GPT-2, GPT-J, BLOOM, Pythia, and LLaMA-2 on diverse linguistic phenomena from BLIMP.
Evidence
correlational
Caveat
Convergence is described as 'approximately monotonic' rather than strictly monotonic; primary evidence is qualitative visualizations supplemented by mean KL divergence across layers, with no specific divergence values reported in the text.