论文标题
用代表性不足的语言特征仔细地增强伯特
Augmenting BERT Carefully with Underrepresented Linguistic Features
论文作者
论文摘要
事实证明，来自变形金刚（BERT）基于序列的序列分类模型的微调双向编码器表示有效检测人类言语转录的阿尔茨海默氏病（AD）。但是，先前的研究表明，通过使用其他信息来增强模型，可以提高伯特在各种任务上的绩效。在这项工作中，我们使用探测任务作为内省技术来识别伯特各个层中代表不大的语言信息，但对于广告检测任务很重要。我们补充了这些语言特征，在这些语言特征中，发现BERT的表示与外部手工制作的特征不足，并表明，与这些功能结合使用的共同微调Bert可以将AD分类的性能提高到5 \％，而不是单独调整的Bert。
Fine-tuned Bidirectional Encoder Representations from Transformers (BERT)-based sequence classification models have proven to be effective for detecting Alzheimer's Disease (AD) from transcripts of human speech. However, previous research shows it is possible to improve BERT's performance on various tasks by augmenting the model with additional information. In this work, we use probing tasks as introspection techniques to identify linguistic information not well-represented in various layers of BERT, but important for the AD detection task. We supplement these linguistic features in which representations from BERT are found to be insufficient with hand-crafted features externally, and show that jointly fine-tuning BERT in combination with these features improves the performance of AD classification by upto 5\% over fine-tuned BERT alone.
