论文标题
回声州神经机器翻译
Echo State Neural Machine Translation
论文作者
论文摘要
我们提出了灵感来自回声状态网络（ESN）的神经机器翻译（NMT）模型，该模型名为Echo State NMT（ESNMT），其中编码器和解码器层的权重随机生成，然后整个训练整个训练。我们表明，即使采用了非常简单的模型构建和培训程序，ESNMT也已经达到了完全可训练的基线的70-80％。我们研究了储层的光谱半径是表征模型的关键数量，它决定了模型行为。我们的发现表明，即使对于复杂的序列到序列预测NLP任务，随机网络也可以很好地工作。
We present neural machine translation (NMT) models inspired by echo state network (ESN), named Echo State NMT (ESNMT), in which the encoder and decoder layer weights are randomly generated then fixed throughout training. We show that even with this extremely simple model construction and training procedure, ESNMT can already reach 70-80% quality of fully trainable baselines. We examine how spectral radius of the reservoir, a key quantity that characterizes the model, determines the model behavior. Our findings indicate that randomized networks can work well even for complicated sequence-to-sequence prediction NLP tasks.
