IC-484Re-scaling the first 2-3 principal components of Llama-3.1-8B token representations causally shifts next-token predictions toward the target graph position
Core Francisco Park, Andrew Lee, Ekdeep Singh Lubana, Yongyi Yang, Maya Okawa, Kento Nishi, Martin Wattenberg, Hidenori Tanaka
The authors intervene on Llama-3.1-8B's activations by re-scaling the first 2 or 3 principal components of a token's representation to match another token's mean representation, effectively 'moving' the token to a different position in the graph. Across 1,000 randomly generated contexts and all n-1 target positions, this intervention significantly improves hit@1, hit@3, and accumulated probability mass on valid neighbors compared to null or random-vector interventions, across ring, grid, and hexagonal graphs. The effect is strongest with 3 PCs and weakest for the hexagonal graph.
The authors state that the principal components 'have some causal effect on the model's output predictions, but does not provide a full explanation' and that 'our simple re-scaling approach does not perfectly capture a causal relationship between principal components and model predictions.'