论文标题
可以解释和话语主题意识到的神经语言理解
Explainable and Discourse Topic-aware Neural Language Understanding
论文作者
论文摘要
嫁给主题模型和语言模型将语言理解暴露于通过主题以外的句子之外的文档级上下文的更广泛的来源。在语言模型中引入主题语义时，现有方法包含了潜在文档主题比例，而忽略了文档句子中的主题话语。这项工作通过在语言理解中引入可解释的主题表示，从而扩展了研究线，并从相应的每个比例的潜在主题中获得了一组关键术语。此外，我们通过为文档中的每个句子建模主题话语来保留句子主题关联以及文档主题关联。我们提出了一种新颖的神经综合语言模型，该模型在主题和语言模型的联合学习框架中利用了潜在和可解释的主题以及句子级别的主题话语。在一系列任务中进行的实验，例如语言建模，单词感官歧义，文档分类，检索和文本生成表明了拟议模型改善语言理解的能力。
Marrying topic models and language models exposes language understanding to a broader source of document-level context beyond sentences via topics. While introducing topical semantics in language models, existing approaches incorporate latent document topic proportions and ignore topical discourse in sentences of the document. This work extends the line of research by additionally introducing an explainable topic representation in language understanding, obtained from a set of key terms correspondingly for each latent topic of the proportion. Moreover, we retain sentence-topic associations along with document-topic association by modeling topical discourse for every sentence in the document. We present a novel neural composite language model that exploits both the latent and explainable topics along with topical discourse at sentence-level in a joint learning framework of topic and language models. Experiments over a range of tasks such as language modeling, word sense disambiguation, document classification, retrieval and text generation demonstrate ability of the proposed model in improving language understanding.
