论文标题
自动集结：基于自适应学习率调度的深度学习模型
Auto-Ensemble: An Adaptive Learning Rate Scheduling based Deep Learning Model Ensembling
论文作者
论文摘要
结合深度学习模型是促进其在新场景中实施的捷径，该方案可以避免从头开始调整神经网络，损失和培训算法。但是，一旦培训，很难收集足够的准确和多样化的模型。本文提出了自动召集（AE），以收集深度学习模型的检查点，并通过自适应学习率调度算法自动整合它们。这种方法的优点是通过在培训中安排学习率来使模型收敛到各种本地Optima。当LO-CAL最佳解决方案的数量趋于饱和时，所有收集的检查点都用于集合。我们的方法是通用的，可以应用于各种情况。多个数据集和神经网络上的实验结果表明它具有有效且具有竞争力，尤其是在几次学习中。此外，我们提出了一种测量模型之间距离的方法。然后，我们可以确保收集模型的准确性和多样性。
Ensembling deep learning models is a shortcut to promote its implementation in new scenarios, which can avoid tuning neural networks, losses and training algorithms from scratch. However, it is difficult to collect sufficient accurate and diverse models through once training. This paper proposes Auto-Ensemble (AE) to collect checkpoints of deep learning model and ensemble them automatically by adaptive learning rate scheduling algorithm. The advantage of this method is to make the model converge to various local optima by scheduling the learning rate in once training. When the number of lo-cal optimal solutions tends to be saturated, all the collected checkpoints are used for ensemble. Our method is universal, it can be applied to various scenarios. Experiment results on multiple datasets and neural networks demonstrate it is effective and competitive, especially on few-shot learning. Besides, we proposed a method to measure the distance among models. Then we can ensure the accuracy and diversity of collected models.
