IC-731CLIP (ViT-B/32) achieves only 17.5 recall on video-text temporal alignment because it was trained on images and lacks video dynamics

Yijie Lin, Jie Zhang, Zhenyu Huang, Jia Liu, zujie wen, Xi Peng

SourceMulti-granularity Correspondence Learning from Long-term Noisy Videos

The paper evaluates CLIP ViT-B/32 on the HTM-Align dataset (80 videos, 49k manually annotated sentences) for the task of aligning text sentences to the correct video frames. CLIP achieves a recall of only 17.5, substantially below video-specific models such as VideoCLIP (44.4) and TempCLR (44.1). The authors attribute this large gap to CLIP having been trained exclusively on image-text pairs, meaning it lacks the ability to capture temporal dynamics in video. This is presented as evidence that image-only pre-training is insufficient for video-text temporal alignment.

Evidence

correlational

Key metric

recall 17.5 on HTM-Align (80 videos, 49k sentences), compared to VideoCLIP 44.4, TempCLR 44.1, TAN-32frame 41.1, TAN-64frame 49.2

Caveat

The authors use the word 'possibly' when attributing the gap to image-only training, indicating this is a hypothesis rather than a confirmed causal mechanism. CLIP was not fine-tuned on HowTo100M like the other models, so the comparison also reflects a difference in training data and procedure.

Model

CLIP / CLIP-ViT (LC)

Concepts

Failure mode

Related work

VideoCLIP [compared-to]

Extraction

automatic-extraction