The paper introduces a Retriever-Dictionary (RD) module that enhances the performance of several pre-existing object detection models by enabling them to leverage external dataset knowledge during training. The module was integrated into YOLOv7, YOLOv9, Faster R-CNN, and Deformable DETR, with the dictionary initialized using different encoders (a vision model, a vision-language model, and a large language model). The primary finding is that incorporating this module consistently improves the mean Average Precision (mAP) of these models on the COCO 2017 validation set, with less than a 1% increase in model parameters for YOLO-based models.
The results for YOLOv7 and YOLOv9 are slightly lower than originally reported in their respective papers, which the authors attribute to differences in the number of GPUs used and the reduced effectiveness of batch normalization with smaller batch sizes.