DomainNet

anchor

Findings

• IC-1017Vision and language models pre-trained on noisy data exhibit degraded OOD transfer that is partially recoverable via SVD-based feature-space regularization [eval]
• IC-1099AdamW-pretrained vision models (ViTs, ConvNeXt) have disproportionately large embedding-layer gradients at initialization, causing SGD fine-tuning to degrade OOD accuracy by up to 15% relative to AdamW [eval]
• IC-147CLIP's OOD performance on rendition domains is largely an artifact of domain contamination in its web-scale training data [eval]