SourceSparse Autoencoders Reveal Temporal Difference Learning in Large Language Models
The authors replicate the two-step, grid world, and graph learning analyses on Gemma-2-27B and Qwen-2.5-72B. For both models, SAE features show strong correlations with TD-driven value and error signals that outperform alternative models (myopic, transition matrix, surprise). Behavioral patterns are qualitatively similar, though neither model performs the two-step task as well as Llama 3 70B, so RL model fits are less accurate. In the graph task, both models show strong SR correlations and community structures emerging over transformer blocks.