IC-442Routing each evaluation instance to the model strongest on its relevant skills yields a 3.2% accuracy gain over the best single model, with 3.5-6.8% gains on MMLU Pro
Mazda Moayeri, Vidhisha Balachandran, Varun Chandrasekaran, Safoora Yousefi, Thomas FEL, Soheil Feizi, Besmira Nushi, Neel Joshi, Vibhav Vineet
The paper tests whether skill-slice insights generalize to held-out data by routing each instance to the model with the highest weighted skill-slice accuracy for that instance. Across 12 datasets, routing improves accuracy by up to 3.2% over each frontier model alone. On MMLU Pro (language-only, while the reference corpus is mostly multimodal), gains are 3.5 to 6.8%. Skill-wise accuracies computed over multimodal benchmarks correlate strongly (r ≥ 0.79, p < 1e-7) with accuracies on the language-only benchmark, supporting generalizability.
Evidence
correlational
Key metric
routing increases accuracy by up to 3.2% compared to each of the frontier models alone on 12 datasets combined, including improvements of 3.5 to 6.8% for mmlu pro; r ≥ 0.79, p < 1e-7 correlation between multimodal and language-only slice accuracies
Caveat
The routing scheme uses a simple weighted average of single-skill accuracies and does not account for added difficulty when skills are used together or for varying instance difficulty within a skill.