论文标题
评估单一RNN作为语法的端到端组成模型
Assessing the Unitary RNN as an End-to-End Compositional Model of Syntax
论文作者
论文摘要
我们表明，LSTM和统一进化的复发神经网络（URN）都可以在两种类型的句法模式上实现令人鼓舞的准确性：无上下文的长距离一致性和轻微的上下文敏感的交叉序列依赖性。这项工作扩展了有关深层无上下文的长距离依赖性的最新实验，结果相似。 URN与LSTM的不同之处在于它们避免了非线性激活函数，并且它们将矩阵乘法应用于编码为单位矩阵的单词嵌入。这使他们可以将所有信息保留在任意距离上的输入字符串中。这也使他们满足了严格的组成性。在应用于NLP的深度学习中，搜索可解释的模型的搜索构成了重大进步。
We show that both an LSTM and a unitary-evolution recurrent neural network (URN) can achieve encouraging accuracy on two types of syntactic patterns: context-free long distance agreement, and mildly context-sensitive cross serial dependencies. This work extends recent experiments on deeply nested context-free long distance dependencies, with similar results. URNs differ from LSTMs in that they avoid non-linear activation functions, and they apply matrix multiplication to word embeddings encoded as unitary matrices. This permits them to retain all information in the processing of an input string over arbitrary distances. It also causes them to satisfy strict compositionality. URNs constitute a significant advance in the search for explainable models in deep learning applied to NLP.
