Self-Instruct

anchor

Findings

• IC-1194GPT-3.5-turbo, GPT-4, and GPT-3.5-turbo-0613 exhibit 50-58% inconsistency between their ratings and rankings feedback on the same response pairs [source]
• IC-682GPT-3.5 and GPT-4 achieve F1 scores of 0.5820 and 0.6180 respectively on pairwise response quality evaluation against human annotations [context]
• IC-986Most LLMs lack tool usage awareness, with only ChatGPT exceeding 70% F1 in zero-shot evaluation [eval]