IC-669DINOv2's features, when clustered, produce smooth semantic regions but lack instance-level boundary delineation

Cheng Shi, Sibei Yang

SourceThe Devil is in the Object Boundary: Towards Annotation-free Instance Segmentation using Foundation Models

The paper compares k-means clustering on DINOv2's middle-layer features with CLIP's under identical settings. DINOv2's clustering results appear smoother with larger contiguous blocks sharing similar semantics, but they do not delineate individual object boundaries. On COCO val2017, DINOv2 achieves only AP 0.8 (class-agnostic) and 0.2 (class-aware) for instance segmentation, far below SAM and CLIP-based methods. The authors attribute this to DINOv2's self-supervised pretraining objective, which encourages global semantic coherence rather than instance-level discrimination.

Evidence
observational
Key metric
DINOv2: AP 0.8 (class-agnostic), AP 0.2 (class-aware) on COCO val2017 (Table 1)
Model
DINOv2
Datasets
MS COCO / COCO / COCO 2014 / COCO 2017 / COCO 20k / COCO-it / COCO-wl [eval]
Methods
K-means clustering [primary]
Related findings
IC-667, IC-668
Extraction
automatic-extraction