Modelpedia
Work in progress
About
Findings
Models
Concepts
Methods
Datasets
Sources
Light
Dark
Improving Complex Reasoning with Dynamic Prompt Corruption: A Soft Prompt Optimization Approach
2025-01-22
· ICLR 2025 Poster ·
anchor
Findings
IC-402
In LLaMA3-8B, LLaMA2-13B, and Mistral-7B, soft-prompt information flow peaks in shallow layers (2–10) and reasoning correctness depends on whether deeper layers redirect attention away from soft prompts to earlier reasoning steps