In the multi-tool selection task, models must choose the correct pair of tools from a candidate list. ChatGPT achieves 88.28% 2/2 CSR while ChatGLM2 manages only 20.20%, a gap of nearly 70 points. A common error is omitting a tool: Vicuna-33b selects only one correct tool in 48.49% of cases under the multi-choice prompt. When the prompt is changed to force exactly two selections (one-choice), Vicuna-33b's CSR jumps to 91.15% and Vicuna-7b improves by over 20 points, indicating that these models rely on the explicitly stated count rather than reasoning about which tools are actually needed.
The multi-tool test set is small (497 queries from TOOLE), and the authors note that the 15 most similar tools to the ground-truth pair are excluded from the candidate list to avoid excessive difficulty.