论文标题
单词嵌入和神经网络中的语法表示形式 - 一项调查
Syntax Representation in Word Embeddings and Neural Networks -- A Survey
论文作者
论文摘要
接受自然语言处理任务的神经网络捕获语法，即使没有作为监督信号提供。这表明句法分析对于人工智能系统中语言的低估至关重要。该概述论文涵盖了评估不同神经网络体系结构单词表示中包含的句法信息量的方法。我们主要总结有关语言建模任务和神经机器翻译系统和多语言语言模型的多语言数据的英语单语言数据。我们描述了哪些预训练的模型和语言的表示最适合转移到句法任务中。
Neural networks trained on natural language processing tasks capture syntax even though it is not provided as a supervision signal. This indicates that syntactic analysis is essential to the understating of language in artificial intelligence systems. This overview paper covers approaches of evaluating the amount of syntactic information included in the representations of words for different neural network architectures. We mainly summarize re-search on English monolingual data on language modeling tasks and multilingual data for neural machine translation systems and multilingual language models. We describe which pre-trained models and representations of language are best suited for transfer to syntactic tasks.
