GEODE

anchor

Findings

• IC-1469Progress on standard ImageNet generalization benchmarks is 2.5x faster than progress on crowdsourced global data (DollarStreet, GEODE) across 98 vision models [eval]
• IC-1470Geographic disparities (Europe-Africa accuracy gap) are large across all 98 models and have more than tripled between least and best performing models on DollarStreet [eval]
• IC-1471Common robustness interventions (AugMix, CutMix, Deep AugMix, texture debiasing, antialiasing) and scaling of data or model size do not resolve geographic disparities in released vision models [eval]
• IC-1472DINOv2 (86M parameters) achieves the smallest GEODE geographic disparity (2.46% Europe-Africa gap) among all 98 models in the testbed [eval]