IC-078GPT-4's self-verification loop causes performance collapse due to high false negative rates in binary verification

Kaya Stechly, Karthik Valmeekam, Subbarao Kambhampati

SourceOn the self-verification limitations of large language models on reasoning and planning tasks

When GPT-4 is used as both the solution generator and the verifier in an iterative self-critique loop, performance drops below the standard single-prompt baseline in most domains. The primary mechanism is a high false negative rate: GPT-4 rejects its own correct answers, causing the system to discard valid solutions and time out on later, worse generations. In graph coloring the false negative rate reaches 95.8% and in mystery blocksworld 97.09%. Even adding chain-of-thought to the verification prompt improves accuracy (e.g., Game of 24 from 87.0% to 98.8%) but does not close the gap to a sound verifier.

Evidence
correlational
Key metric
Table 2: verification accuracy and FNR — game of 24: 87.0% (3567/4100), FNR 20.7% (213/1029); graph coloring: 72.4% (362/500), FNR 95.8% (113/118); blocksworld: 71.8% (359/500), FNR 15.48% (24/155); mystery blocksworld: 79.6% (398/500), FNR 97.09% (100/103). Table 1: LLM+LLM accuracy — game of 24: 3%, graph coloring: 2%, blocksworld: 55%, mystery blocksworld: 0%, versus standard prompting 5%, 16%, 40%, 4% respectively.
Caveat
Results are specific to gpt-4-0613 and three formally verifiable domains. The authors note that blocksworld shows a modest improvement (55% vs 40% baseline) under self-critique, though still far below sound-verifier performance (87%).
Model
GPT-4 / ChatGPT4 / GPT-4 Code Interpreter / GPT-4 Technical Report
Concepts
Failure mode
Datasets
Game of 24 (4nums.com instances 1-1000) [eval], PlanBench [eval]
Methods
Self-Consistency / Self-consistency prompting / Wang et al. 2023 (self-consistency) / Wang et al. 2023b (Self-Consistency) [compared-to]
Related work
Self-Refine [compared-to]
Related findings
IC-079, IC-080
Extraction
automatic-extraction