Modelpedia
Work in progress
About
Findings
Models
Concepts
Methods
Datasets
Sources
Light
Dark
Neuron-based Multifractal Analysis of Neuron Interaction Dynamics in Large Models
2025-01-22
· ICLR 2025 Poster ·
anchor
Findings
IC-471
Pythia models exceeding 100M parameters show a consistent leftward shift of the multifractal spectrum (increasing regularity) during training that is absent in the 14M and 31M variants
IC-472
The degree of emergence metric derived from Pythia's internal structure positively correlates with benchmark performance across training epochs
IC-473
ResNet-18 lacks a clear multifractal structure while ResNet-152 shows one with irregular shifts, and a 160M diffusion model exhibits lower degree of emergence than Pythia 160M