NuNER: Entity Recognition Encoder Pre-training via LLM-Annotated Data

Large Language Models (LLMs) have shown impressive abilities in dataannotation, opening the way for new approaches to solve classic NLP problems.In this paper, we show how to use LLMs to create NuNER, a compact languagerepresentation model specialized in the Named Entity Recognition (NER) task.NuNER can be fine-tuned to solve downstream NER problems in a data-efficientway, outperforming similar-sized foundation models in the few-shot regime andcompeting with much larger LLMs. We find that the size and entity-typediversity of the pre-training dataset are key to achieving good performance. Weview NuNER as a member of the broader family of task-specific foundationmodels, recently unlocked by LLMs.

Further reading#

• Access Paper in arXiv.org