Knowledge Distillation

anchor

Train a smaller model to match a larger model's output distribution rather than the hard labels.

Findings

• IC-006The Retriever-Dictionary module improves object detection accuracy of YOLOv7, YOLOv9, Faster R-CNN, and Deformable DETR on COCO 2017 [compared-to]