Modelpedia
Work in progress
About
Findings
Models
Concepts
Methods
Datasets
Sources
Light
Dark
Quantifying Generalization Complexity for Large Language Models
2025-01-22
· ICLR 2025 Poster ·
anchor
Findings
IC-433
LLMs exhibit a non-monotonic ID-OOD performance gap (generalization valley) that peaks at intermediate task complexity
IC-434
The critical complexity at which LLMs over-rely on memorization shifts to higher task difficulty as model size increases
IC-435
Mistral-7B employs a less efficient algorithmic strategy (O(n²)) than Llama-3-8B (O([n², n³])) on probe tasks with multiple solution complexities