IC-1201CLIP-ViT-L/14 image features support 200-way zero-shot EEG-based object recognition better than ViT-B/16 or ResNet-50 features when used as a frozen encoder in a contrastive learning framework
The paper uses three released image encoders as frozen feature extractors in a contrastive learning pipeline that aligns image features with EEG features via cosine similarity. CLIP-ViT-L/14 (pre-trained on 400M image-text pairs) achieves the highest average top-1 accuracy of 13.8% and top-5 of 39.5% on the 200-way zero-shot task. Pre-trained ViT-B/16 achieves 12.1% top-1 (1.7% lower, p > 0.05), and pre-trained ResNet-50 achieves 8.8% top-1 (6.9% lower, p < 0.001). Notably, pre-training on ImageNet-21k+1k improves ViT-B/16 by 4.2% over random initialization (p < 0.01), but pre-training on ImageNet-1k degrades ResNet-50 by 1.9% (p < 0.05). In the appendix, EVA-CLIP pre-trained on LAION-2B further improves to 18.7% top-1 and 47.6% top-5.
The image encoder is frozen and only the EEG encoder is trained; the comparison isolates the image encoder's contribution but the overall system performance also depends on the authors' TSConv EEG encoder. The ViT vs CLIP gap is not statistically significant (p > 0.05).