IC-402In LLaMA3-8B, LLaMA2-13B, and Mistral-7B, soft-prompt information flow peaks in shallow layers (2–10) and reasoning correctness depends on whether deeper layers redirect attention away from soft prompts to earlier reasoning steps

Sinan Fan, Liang Xie, Chen Shen, Ge Teng, Xiaosong Yuan, Xiaofeng Zhang, Chenxi Huang, Wenxiao Wang, Xiaofei He, Jieping Ye

SourceImproving Complex Reasoning with Dynamic Prompt Corruption: A Soft Prompt Optimization Approach

Using saliency score analysis (Hadamard product of attention and gradient), the paper measures layer-by-layer information flow from soft prompt tokens to question and rationale tokens in three released LLMs. A strong prompt-to-question and prompt-to-rationale flow is concentrated in layers 2 through 10. In correct-answer cases, deeper layers shift attention from soft prompts to earlier rationale steps and the question; in incorrect cases, the latter reasoning steps in deep layers maintain strong attention to soft prompts. This pattern was validated on 100 randomly sampled instances, and the relationship between the last-layer prompt-to-rationale flow intensity (sif_p) and correctness was confirmed on 50 correct and 50 incorrect examples. The practical consequence is that vanilla prompt tuning yields only marginal gains on complex reasoning (e.g., LLaMA3-8B GSM8K: 64.9% pretrained to 65.5% with PT, a 0.6% increase).

Evidence

correlational

Key metric

Saliency peaks between layer 2 and layer 10; 100 randomly sampled instances for correlation validation; 50 correct + 50 incorrect for sif_p analysis; LLaMA3-8B GSM8K: pretrained 64.9%, prompt tuning 65.5% (0.6% gain); LLaMA2-13B GSM8K: pretrained 29.5%, PT 38.1%; Mistral-7B GSM8K: pretrained 37.9%, PT 49.5%

Caveat

The analysis is limited to three decoder-only LLMs and three math-reasoning datasets; the authors note that the inference cost of identifying erroneous information flow patterns requires a complete reasoning pass, limiting practical applicability.

Model

Llama 3 8B, Llama 2 / Llama 2 base Llama 2 13B, Mistral 7B / Mistral / Mistral 3 7B / Mistral-0.2-7B / Mistral-v0.1

Concepts

Depth-dependent structure, Failure mode

Datasets

GSM8K [eval], AQUA / AQUA-RAT [eval]

Methods

Prompt Tuning / Soft prompt tuning [compared-to]

Extraction

automatic-extraction