论文标题
对低资源命名实体识别的预训练的编码器的比较研究
A Comparative Study of Pre-trained Encoders for Low-Resource Named Entity Recognition
论文作者
论文摘要
预先训练的语言模型（PLM）是在持续进行特定于任务的室外数据或对内域数据进行微调的持续预培训时，是命名实体识别（NER）方法的有效组成部分。但是，它们在没有此类数据的低资源场景中的性能仍然是一个悬而未决的问题。我们介绍了一个编码器评估框架，并将其用于系统地比较最先进的预培训表示形式在低资源NER任务上的性能。我们分析了通过不同策略，模型架构，中间任务微调和对比度学习的广泛编码器。我们在英语和德语的十个基准NER数据集中的实验结果表明，编码器的性能差异很大，这表明需要仔细评估编码器的编码器选择。
Pre-trained language models (PLM) are effective components of few-shot named entity recognition (NER) approaches when augmented with continued pre-training on task-specific out-of-domain data or fine-tuning on in-domain data. However, their performance in low-resource scenarios, where such data is not available, remains an open question. We introduce an encoder evaluation framework, and use it to systematically compare the performance of state-of-the-art pre-trained representations on the task of low-resource NER. We analyze a wide range of encoders pre-trained with different strategies, model architectures, intermediate-task fine-tuning, and contrastive learning. Our experimental results across ten benchmark NER datasets in English and German show that encoder performance varies significantly, suggesting that the choice of encoder for a specific low-resource scenario needs to be carefully evaluated.
