论文标题
埃斯特伯特：爱沙尼亚人的特定语言特定语言
EstBERT: A Pretrained Language-Specific BERT for Estonian
论文作者
论文摘要
本文介绍了Estbert，这是爱沙尼亚人的大型基于变压器的特定语言BERT模型。最近的工作已经评估了关于爱沙尼亚任务的多语言BERT模型，并发现它们表现优于基准。尽管如此，基于对其他语言的现有研究，预计特定于语言的BERT模型将改善多语言的BERT模型。我们首先描述了Estbert预处理过程，然后基于备键式Estbert的多个NLP任务，包括POS和形态标记，命名实体识别和文本分类，介绍模型的结果。评估结果表明，基于Estbert的模型在六个任务中的五个任务上都优于多语言BERT模型，从而提供了进一步的证据，即即使有多语言模型，也可以证明培训语言特定的BERT模型仍然有用。
This paper presents EstBERT, a large pretrained transformer-based language-specific BERT model for Estonian. Recent work has evaluated multilingual BERT models on Estonian tasks and found them to outperform the baselines. Still, based on existing studies on other languages, a language-specific BERT model is expected to improve over the multilingual ones. We first describe the EstBERT pretraining process and then present the results of the models based on finetuned EstBERT for multiple NLP tasks, including POS and morphological tagging, named entity recognition and text classification. The evaluation results show that the models based on EstBERT outperform multilingual BERT models on five tasks out of six, providing further evidence towards a view that training language-specific BERT models are still useful, even when multilingual models are available.
