论文标题
通过自然进化策略和随机梯度下降来学习深度多任务学习的重量共享
Learned Weight Sharing for Deep Multi-Task Learning by Natural Evolution Strategy and Stochastic Gradient Descent
论文作者
论文摘要
在深度多任务学习中，任务之间共享特定于任务的网络的权重，以提高每个单个单一的性能。由于在两层之间要共享的权重的问题很难回答，因此人类设计的体系结构通常共享除了最后一个特定于任务的层之外的所有内容。在许多情况下，这种简单的方法严重限制了性能。取而代之的是，我们提出了一种算法，以了解共享的一组权重和特定于任务的层之间的分配。为了优化非差异性分配，同时训练可区分的权重，学习是通过自然进化策略和随机梯度下降的结合进行学习。最终结果是特定于任务的网络，它们共享权重但允许独立推断。它们比基线的测试错误以及有关三个多任务学习数据集的文献的方法较低。
In deep multi-task learning, weights of task-specific networks are shared between tasks to improve performance on each single one. Since the question, which weights to share between layers, is difficult to answer, human-designed architectures often share everything but a last task-specific layer. In many cases, this simplistic approach severely limits performance. Instead, we propose an algorithm to learn the assignment between a shared set of weights and task-specific layers. To optimize the non-differentiable assignment and at the same time train the differentiable weights, learning takes place via a combination of natural evolution strategy and stochastic gradient descent. The end result are task-specific networks that share weights but allow independent inference. They achieve lower test errors than baselines and methods from literature on three multi-task learning datasets.
