COMET

anchor

Findings

• IC-1236Among 7B LLMs, Llama-2-7b achieves the best zero-shot COMET scores in both translation directions, while MPT-7b leads in BLEU for en-to-xx [eval]
• IC-1237Llama-2-7b's pre-existing translation knowledge is diluted by large amounts of parallel data, causing COMET to decline after 100k examples [eval]
• IC-1238Llama-2-13b produces off-target non-translation outputs in zero-shot English-to-foreign-language translation [eval]