The paper evaluates 20 open- and closed-source LLMs on 586 multi-turn interaction tasks spanning reasoning, code generation, and decision-making. Every model shows absolute performance gains of 1-8% per additional tool-use turn (k=1 to k=5) and 2-17% from GPT-4-simulated natural language feedback. Closed-source models (GPT-3.5-turbo-0613, Claude-instant-1) show the steepest per-turn improvement slopes (+8.2 and +8.0), while the best open-source model (Lemur-v1-70b SIFT) still lags the best closed-source model by 8.7% in absolute success rate with feedback.
Evidence
correlational
Key metric
performance gains of 1-8% for each turn of tool use and 2-17% with natural language feedback; slopes range +1.1 (CodeLlama-7b base) to +8.2 (GPT-3.5-turbo-0613); delta_feedback ranges +1.7 (Llama-2-7b RLHF) to +17.2 (CodeLlama-7b SIFT)
Caveat
GPT-4-simulated feedback may not cover all possible human responses, particularly in tasks involving nuanced judgments of human values. Metrics may not fully assess interaction quality beyond outcomes.