Aggregating neuron paths over all 50 images of each ImageNet class, the paper finds that at most layers, a small set of neurons is selected with significantly higher frequency for a given class (intra-class clustering). Furthermore, computing cosine similarity between per-class neuron utilization matrices reveals that semantically similar classes (e.g., nile crocodile and alligator lizard, sim 0.501) have higher neuron path similarity than dissimilar classes (e.g., nile crocodile and mileometer, sim 0.144). This pattern holds across ViT-B/16, ViT-B/32, ViT-L/32, and MAE-B/16 as shown in appendix visualizations.
Evidence
observational
Key metric
Cosine similarity examples (ViT-B/16): nile crocodile–alligator lizard 0.501, nile crocodile–komodo lizard 0.623, nile crocodile–mileometer 0.144, nile crocodile–smoothing iron 0.178
Caveat
The analysis is limited to 50 images per class from the ImageNet validation set, and the similarity is computed on neuron selection frequency rather than activation magnitude.