SourceUniversalNER: Targeted Distillation from Large Language Models for Open Named Entity Recognition
InstructUIE, a multi-task instruction-tuned system based on FLAN-T5 11B that uses supervised NER examples, is evaluated in both in-domain and out-of-domain settings. In the in-domain setting (Table 2, 20 datasets from Wang et al. 2023a), it achieves 81.16% average F1. In the out-of-domain setting (Table 3, excluding CrossNER and MIT datasets from training), it achieves 49.4% average F1. The paper notes a critical bug in InstructUIE's original evaluation script that inflates F1 for passages without entities, and re-evaluated using the released checkpoint.