Modelpedia
Work in progress
About
Findings
Models
Concepts
Methods
Datasets
Sources
Light
Dark
Toward Understanding In-context vs. In-weight Learning
2025-01-22
· ICLR 2025 Poster ·
anchor
Findings
IC-324
Fine-tuning Gemini Nano 1 on 8 memorization examples causes it to override in-context predictions with in-weight predictions in 2 of 8 cases, while the base model always follows in-context predictions