Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verification

2024-01-16 · ICLR 2024 poster · anchor

Findings

IC-1167GPT-4 Code Interpreter's mathematical reasoning accuracy is positively correlated with code usage frequency, with its iterative code generation and self-debugging mechanism as the primary driver of its 69.69% zero-shot MATH accuracy
IC-1168For GPT-4 Code Interpreter, code-based self-verification improves MATH accuracy to 73.54% while natural language self-verification slightly degrades it to 69.29% relative to the 69.69% base
IC-1169CodeLlama-7B and CodeLlama-34B show improvement with CSV prompting on GSM8K and MATH but at much lower absolute accuracy than GPT-4 Code Interpreter