SourceLarge Language Model Cascades with Mixture of Thought Representations for Cost-Efficient Reasoning
When GPT-3.5-turbo fails on a hard reasoning question under chain-of-thought prompting, re-prompting it with a different set of CoT demonstrations typically yields the same incorrect answer. In contrast, program-of-thought prompting tends to produce a different (or correct) answer. Among cases where CoT fails, 43% are solved by PoT and 57% receive a different wrong answer from PoT. This error-correlation structure is what makes mixing CoT and PoT (MOT) more effective at detecting model uncertainty than either representation alone.