UniversalNER: Targeted Distillation from Large Language Models for Open Named Entity Recognition

2024-01-16 · ICLR 2024 poster · anchor

Findings

IC-1463ChatGPT achieves 34.9% average F1 on zero-shot NER across 43 datasets spanning 9 domains
IC-1464Vicuna-7B and Vicuna-13B achieve only 14.2% and 18.0% average F1 on zero-shot NER, trailing ChatGPT by over 20 points
IC-1465InstructUIE-11B achieves 81.16% average F1 on 20 in-domain NER datasets and 49.4% on out-of-domain evaluation