IC-1165ProtoPFormer achieves 42.2% attribute identification accuracy on CUB-200-2011 in a 7-rater human evaluation
DIPANJYOTI PAUL, Arpita Chowdhury, Xinqi Xiong, Feng-Ju Chang, David Edward Carlyn, Samuel Stevens, Kaiya L Provost, Anuj Karpatne, Bryan Carstens, Daniel Rubenstein, Charles Stewart, Tanya Berger-Wolf, Yu Su, Wei-Lun Chao
The authors evaluated ProtoPFormer's ability to identify fine-grained visual attributes by showing its prototype activation maps to seven human raters unfamiliar with the work. They randomly selected five images from each of four CUB species (20 images total) and asked raters to list attributes captured by the maps. An attribute was counted as detected if more than half of the raters identified it. ProtoPFormer achieved 42.2% attribute identification accuracy, compared to 74.7% for the authors' INTR model on the same protocol.
Evidence
observational
Key metric
attribute identification accuracy 42.2% (ProtoPFormer) vs 74.7% (INTR), 20 CUB images, 7 human raters
Caveat
Very small sample: only 20 images across 4 species, evaluated by 7 raters; the evaluation protocol was designed by the authors specifically for this comparison