Modelpedia
Work in progress
About
Findings
Models
Concepts
Methods
Datasets
Sources
Light
Dark
Competition Dynamics Shape Algorithmic Phases of In-Context Learning
2025-01-22
· ICLR 2025 Spotlight ·
anchor
Findings
IC-302
Llama-3.1 models perform between unigram-inference and bigram-inference on Markov chain ICL, with performance improving monotonically with model scale
IC-303
Explicitly stating the Markovian structure in the prompt significantly improves Llama-3.1-70B's next-state prediction on the Markov chain task