Modelpedia
Work in progress
About
Findings
Models
Concepts
Methods
Datasets
Sources
Light
Dark
WILDS-FMOW
anchor
Findings
IC-1099
AdamW-pretrained vision models (ViTs, ConvNeXt) have disproportionately large embedding-layer gradients at initialization, causing SGD fine-tuning to degrade OOD accuracy by up to 15% relative to AdamW
[eval]