The paper introduces a 'second-order lens' that traces each neuron's contribution through subsequent attention heads to the output. Mean-ablating all second-order effects in a given layer and measuring zero-shot ImageNet accuracy shows that layers 8–10 produce the largest accuracy drop, while early layers have negligible effect. The same trend holds for the larger ViT-L/14 model, where the drop is smaller but still concentrated in later layers. This contrasts with the first-order (direct) effects of attention heads, which are significant in layers 9–11.
Evidence
interventional
Key metric
Mean-ablating second-order effects in layer 9 (ViT-B/32) drops ImageNet accuracy to 29.6% (Table 1); ablating indirect effects in the same layer drops it only to 52.3%. First PC of second-order effects explains 48.2% of variance vs 11.0% for indirect effects.
Caveat
The paper notes it ignored the effect of neurons on consecutive queries and keys in the attention mechanism, and did not analyze mutual effects between neurons in the same or different layers.