论文标题
PFGE：DNN的简约快速几何结合
PFGE: Parsimonious Fast Geometric Ensembling of DNNs
论文作者
论文摘要
集合方法通常用于增强机器学习模型的概括性能。但是，由于训练深度神经网络（DNNS）的合奏所需的高计算间接费用，他们在深度学习系统中提出了挑战。最近的进步，例如快速几何结合（FGE）和快照集合，通过与单个模型同时通过训练模型合奏来解决此问题。尽管如此，与基于单模的方法相比，这些技术仍然需要进行测试时间推理的额外记忆。在本文中，我们提出了一种名为Parsonious FGE（PFGE）的新方法，该方法采用了通过连续的随机重量平均程序产生的高性能DNN的轻量级合奏。我们对CIFAR- {10,100}和跨各种现代DNN体系结构的Imagenet数据集的实验结果表明，与以前的方法相比，PFGE在不影响概括性能的情况下达到了5倍的存储效率。对于有兴趣的人，我们的代码可在https://github.com/zjlab-ammi/pfge上找到。
Ensemble methods are commonly used to enhance the generalization performance of machine learning models. However, they present a challenge in deep learning systems due to the high computational overhead required to train an ensemble of deep neural networks (DNNs). Recent advancements such as fast geometric ensembling (FGE) and snapshot ensembles have addressed this issue by training model ensembles in the same time as a single model. Nonetheless, these techniques still require additional memory for test-time inference compared to single-model-based methods. In this paper, we propose a new method called parsimonious FGE (PFGE), which employs a lightweight ensemble of higher-performing DNNs generated through successive stochastic weight averaging procedures. Our experimental results on CIFAR-{10,100} and ImageNet datasets across various modern DNN architectures demonstrate that PFGE achieves 5x memory efficiency compared to previous methods, without compromising on generalization performance. For those interested, our code is available at https://github.com/ZJLAB-AMMI/PFGE.
