Discovering Influential Neuron Path in Vision Transformers

2025-01-22 · ICLR 2025 Poster · anchor

Findings

• IC-290Zeroing out or doubling specific FFN neurons identified by the neuron path method causes significant accuracy changes in ViT and MAE models
• IC-291ViT-B/16 and MAE-B/16 exhibit nearly inverted distributions of knowledge neurons across layers despite identical architecture and training data
• IC-292Neuron paths in ViT-B/16 show class-specific neuron clustering and semantic similarity between image categories
• IC-293ViT-B/16 and ViT-B/32 are largely redundant: retaining only top-5 neurons per layer while zeroing all others preserves most classification accuracy