Modelpedia
Work in progress
About
Findings
Models
Concepts
Methods
Datasets
Sources
Light
Dark
Distributional Associations vs In-Context Reasoning: A Study of Feed-forward and Attention Layers
2025-01-22
· ICLR 2025 Poster ·
anchor
Findings
IC-015
Truncating MLP weights in Pythia-1b increases the probability of the correct answer in an Indirect Object Identification task
IC-016
Truncating MLP weights in Pythia-1b increases the probability of the correct answer in a factual recall task
IC-017
Truncating MLP weights improves few-shot Chain-of-Thought reasoning accuracy on GSM8K for Phi-3 and Llama-3.1-8B