SourceMatcher: Segment Anything with One Shot Using All-Purpose Feature Matching
Using each model as the image encoder in the Matcher framework, the paper measures how well their patch-level features support cross-image semantic matching. DINOv2 (ViT-L/14) achieves 52.7% mean mIoU on COCO-20i, 87.0% on FSS-1000, and 79.5 J&F on DAVIS 2017, substantially outperforming CLIP (32.2/77.4/73.9) and MAE (18.8/71.9/69.5). The authors attribute CLIP's weakness to text-image contrastive pre-training limiting pixel-level information, and MAE's weakness to patch-level features confusing surrounding patch information. DINOv2's image-level and patch-level discriminative self-supervised learning is credited with producing all-purpose visual features suitable for precise patch matching.