SourceBrain decoding: toward real-time reconstruction of visual perception
The paper trains a MEG decoder to map brain signals onto pretrained image embeddings and measures retrieval accuracy. Three types of pretrained representations—supervised (VGG-19), contrastive image-text (CLIP-vision), and self-supervised (DINOv2)—all yield strong top-5 retrieval accuracy (~68–70%) on the small test set. When the analysis is repeated on 100-ms sliding windows, DINOv2 in particular shows elevated retrieval performance in windows ending around 150–200 ms after image offset, suggesting its self-supervised representations capture late-stage visual processing in the brain more effectively than the other models tested.