SourceStable Segment Anything Model
The paper systematically evaluates SAM (ViT-Large) across four segmentation datasets using prompts of varying quality: ground-truth boxes, noisy boxes (uniform noise scale 0.4), and point prompts with 1, 3, 5, or 10 points. With noisy box prompts, mIoU drops from 79.5 to 48.8 and the segmentation stability score (ST) is only 39.5; with a single point prompt, mIoU is 43.3 and ST is 45.1. Visualization of the token-to-image cross-attention in the second mask decoder layer reveals that imprecise prompts cause the decoder to activate image features biased toward the background or confined to specific object parts, a phenomenon the authors term attention drift. SAM 2 exhibits the same degradation pattern (mIoU 52.4 with noisy box, 46.7 with 1 point), confirming the failure mode is not specific to the original SAM checkpoint.