SourceMatcher: Segment Anything with One Shot Using All-Purpose Feature Matching
The paper demonstrates that SAM, when used as the sole segmenter (via PerSAM), cannot perform semantic segmentation well. PerSAM achieves only 23.0% mIoU on COCO-20i, 71.2% on FSS-1000, and 11.5% on LVIS-92i in the one-shot setting. The authors attribute this to SAM's inability to provide semantic categories for predicted masks and its tendency to produce multiple ambiguous mask outputs. The paper explicitly states that 'depending solely on SAM results in limited generalization capabilities for semantic tasks.' On one-shot part segmentation, PerSAM also underperforms Matcher by +12.8% on Pascal-Part and +13.5% on PACO-Part.