论文标题
先验知识驱动的标签嵌入插槽填充自然语言理解
Prior Knowledge Driven Label Embedding for Slot Filling in Natural Language Understanding
论文作者
论文摘要
传统的插槽填充自然语言理解（NLU）预测每个单词的一个旋转向量。这种标签表示形式缺乏语义相关模型，从而导致严重的数据稀疏问题，尤其是在将NLU模型适应新领域时。为了解决这个问题，本文提出了一个新型的基于嵌入的插槽填充框架。在这里，使用先验知识为每个插槽构建分布式标签嵌入。研究了三种编码方法，以结合有关插槽的不同类型的先验知识：原子概念，插槽描述和插槽示例。所提出的标签嵌入倾向于共享文本模式并使用不同的插槽标签重用数据。这使得对具有有限数据的自适应NLU有用。同样，由于标签嵌入与NLU模型无关，因此它与几乎所有基于深度学习的插槽填充模型都兼容。提出的方法在三个数据集上进行评估。对单个域和域适应任务进行的实验表明，嵌入标签可以比传统的单速标签表示以及先进的零照片方法来取得重大的性能提高。
Traditional slot filling in natural language understanding (NLU) predicts a one-hot vector for each word. This form of label representation lacks semantic correlation modelling, which leads to severe data sparsity problem, especially when adapting an NLU model to a new domain. To address this issue, a novel label embedding based slot filling framework is proposed in this paper. Here, distributed label embedding is constructed for each slot using prior knowledge. Three encoding methods are investigated to incorporate different kinds of prior knowledge about slots: atomic concepts, slot descriptions, and slot exemplars. The proposed label embeddings tend to share text patterns and reuses data with different slot labels. This makes it useful for adaptive NLU with limited data. Also, since label embedding is independent of NLU model, it is compatible with almost all deep learning based slot filling models. The proposed approaches are evaluated on three datasets. Experiments on single domain and domain adaptation tasks show that label embedding achieves significant performance improvement over traditional one-hot label representation as well as advanced zero-shot approaches.
