Modelpedia
Work in progress
About
Findings
Models
Concepts
Methods
Datasets
Sources
Light
Dark
GAIA: a benchmark for General AI Assistants
2024-01-16
· ICLR 2024 poster ·
anchor
Findings
IC-1243
GPT-4 and GPT-4 Turbo achieve near-zero scores on GAIA level 3 and single-digit to low-double-digit scores on levels 1-2, compared to 87-94% for human annotators
IC-1244
GPT-4's non-zero scores on GAIA web browsing questions are largely due to memorization of intermediate information from training data rather than actual web browsing