The study evaluates the ability of 23 LMMs to perform dense object localization of oracle bone characters on original oracle bone images (from the O2BR dataset) and inked rubbings (from the Yinqiwenyuandetection dataset). The 'where' question required the models to return bounding boxes for each detected character. Performance was measured using mean Intersection over Union (mIoU). All evaluated models performed poorly, with Gemini 1.5 Pro significantly outperforming others by two orders of magnitude, yet still far from public-level human performance.
Gemini 1.5 Pro's superior performance is attributed to its exclusive coordinate return function, which is not a standard capability of other models. Performance is far from human-level, indicating the task remains a significant challenge.