IC-048Existing video LLMs (TimeChat, VTG-LLM, Momentor, Hawkeye) show limited zero-shot video temporal grounding capability and struggle to improve with fine-tuning
The paper evaluates four released video LLMs specifically designed for temporal grounding on three VTG benchmarks in zero-shot settings. All four show substantially lower performance than TRACE, with CIDEr scores on YouCook2 ranging from 1.2 to 31.4 and R@1(IoU=0.7) on Charades-STA from 13.4 to 15.7. After fine-tuning on YouCook2 and Charades-STA, TimeChat and VTG-LLM still struggle to attain satisfactory performance, with CIDEr of 11.0 and 13.4 respectively, far below task-specific models.
Evidence
correlational
Key metric
Zero-shot CIDEr on YouCook2: TimeChat 3.4, VTG-LLM 5.0, Momentor 11.6, Hawkeye 14.5; Zero-shot R@1(IoU=0.7) on Charades-STA: TimeChat 13.4, VTG-LLM 15.7, Hawkeye 14.5; Fine-tuned CIDEr on YouCook2: TimeChat 11.0, VTG-LLM 13.4
Caveat
The paper notes that Valley, VideoChat-embed, and Video-LLaMA results are elaborated from previous studies, and VTimellm (13B) comparison is flagged as unfair due to different model size.