SourceFeature Averaging: An Implicit Bias of Gradient Descent Leading to Non-Robustness in Neural Networks
The authors use a frozen CLIP ViT-B/32 model to extract image embeddings for CIFAR-10 and verify that the resulting representations approximately satisfy the multi-cluster data distribution assumed in their theoretical analysis. Specifically, the correlation between embeddings of images from the same class is significantly higher than between embeddings from different classes, and the class-mean feature vectors are approximately orthogonal. This observation is used to justify applying their feature-averaging theory to a two-layer classifier trained on CLIP embeddings. The result is presented as a setup validation in Appendix H rather than a main contribution.