IC-402In LLaMA3-8B, LLaMA2-13B, and Mistral-7B, soft-prompt information flow peaks in shallow layers (2–10) and reasoning correctness depends on whether deeper layers redirect attention away from soft prompts to earlier reasoning steps
Sinan Fan, Liang Xie, Chen Shen, Ge Teng, Xiaosong Yuan, Xiaofeng Zhang, Chenxi Huang, Wenxiao Wang, Xiaofei He, Jieping Ye
Using saliency score analysis (Hadamard product of attention and gradient), the paper measures layer-by-layer information flow from soft prompt tokens to question and rationale tokens in three released LLMs. A strong prompt-to-question and prompt-to-rationale flow is concentrated in layers 2 through 10. In correct-answer cases, deeper layers shift attention from soft prompts to earlier rationale steps and the question; in incorrect cases, the latter reasoning steps in deep layers maintain strong attention to soft prompts. This pattern was validated on 100 randomly sampled instances, and the relationship between the last-layer prompt-to-rationale flow intensity (sif_p) and correctness was confirmed on 50 correct and 50 incorrect examples. The practical consequence is that vanilla prompt tuning yields only marginal gains on complex reasoning (e.g., LLaMA3-8B GSM8K: 64.9% pretrained to 65.5% with PT, a 0.6% increase).
Evidence
correlational
Key metric
Saliency peaks between layer 2 and layer 10; 100 randomly sampled instances for correlation validation; 50 correct + 50 incorrect for sif_p analysis; LLaMA3-8B GSM8K: pretrained 64.9%, prompt tuning 65.5% (0.6% gain); LLaMA2-13B GSM8K: pretrained 29.5%, PT 38.1%; Mistral-7B GSM8K: pretrained 37.9%, PT 49.5%
Caveat
The analysis is limited to three decoder-only LLMs and three math-reasoning datasets; the authors note that the inference cost of identifying erroneous information flow patterns requires a complete reasoning pass, limiting practical applicability.