Modelpedia
Work in progress
About
Findings
Models
Concepts
Methods
Datasets
Sources
Light
Dark
MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback
2024-01-16
· ICLR 2024 poster ·
anchor
Findings
IC-1330
All 20 evaluated LLMs improve in multi-turn task-solving with additional tool-use turns and GPT-4-simulated language feedback
IC-1331
SIFT and RLHF variants of CodeLlama and Llama-2 perform worse than their base counterparts in multi-turn interaction
IC-1332
Vicuna-v1.5 and CodeLlama-34b-instruct produce format-breaking artifacts (escaped underscores, [python] tags) in 30-100% of code instances due to training data contamination