IC-192CLIP's contrastive image-text training objective hinders its ability to rank or order images, yielding near-chance performance on ranking tasks in both zero-shot and fine-tuned settings

Wei-Hsiang Yu, Yen-Yu Lin, Ming-Hsuan Yang, Yi-Hsuan Tsai

SourceRanking-aware adapter for text-driven image ordering with CLIP

The paper measures CLIP (ConvNeXt-L, LAION-5B) on four ranking tasks: facial age estimation, historical image dating, object count sorting, and image quality/aesthetics assessment. In zero-shot mode, CLIP achieves only 43.3% accuracy (MAE 0.80) on ADIENCE age estimation, 26.1% accuracy (MAE 1.48) on HCI dating, and PLCC 0.026 / SRCC 0.001 on object count sorting. Even after fine-tuning with standard contrastive objectives, CLIP reaches only PLCC 0.251 / SRCC 0.422 on object counting and PLCC 0.245 / SRCC 0.216 on KONIQ-10k. The paper attributes this failure to CLIP's training objective, which matches image content to sentences and captures global information but does not learn cross-image relational structure needed for ordering.

Evidence
correlational
Key metric
Zero-shot CLIP: ADIENCE accuracy 43.3 (3.6)%, MAE 0.80 (0.02); HCI accuracy 26.1 (0.6)%, MAE 1.48 (0.03); object count PLCC 0.026, SRCC 0.001; KONIQ-10k PLCC 0.245, SRCC 0.216; AVA PLCC 0.162, SRCC 0.160. Fine-tuned CLIP: object count PLCC 0.251, SRCC 0.422; KONIQ-10k PLCC 0.245, SRCC 0.216; AVA PLCC 0.162, SRCC 0.160.
Caveat
The paper does not ablate the training objective in isolation to confirm causality; the attribution to the contrastive objective is stated as motivation rather than tested experimentally. The fine-tuned CLIP baseline uses the same contrastive objective, so it is unclear whether a different fine-tuning objective would close the gap.
Model
CLIP / CLIP-ViT (LC)
Concepts
Failure mode
Datasets
ADIENCE [eval], HCI (Historical Colored Images) [eval], COCO-REM [eval], KONIQ-10K [eval], AVA [eval]
Methods
OpenCLIP [eval]
Related work
OrdinalCLIP [compared-to]
Extraction
automatic-extraction