论文标题
DADA：可区分的自动数据增强
DADA: Differentiable Automatic Data Augmentation
论文作者
论文摘要
数据增强（DA）技术旨在提高数据可变性，从而以更好的概括来训练深层网络。开创性的自动仪通过增强学习来自动搜索最佳DA政策。但是，自动计算在计算上非常昂贵，从而限制了其广泛的适用性。后续工作，例如基于人口的增强（PBA）和快速自动仪的效率，但其优化速度仍然是瓶颈。在本文中，我们提出了可区分的自动数据增强（DADA），该数据大大降低了成本。 DADA通过Gumbel-Softmax放松离散的DA策略选择，以解决可区分的优化问题。此外，我们引入了公正的梯度估计器，放松，导致一种有效而有效的一通优化策略，以学习有效而准确的DA策略。我们对CIFAR-10，CIFAR-100，SVHN和Imagenet数据集进行了广泛的实验。此外，我们证明了自动DA在下游检测问题的预训练中的价值。结果表明，我们的DADA至少比最先进的速度快一个数量级，同时达到了非常可比的精度。该代码可在https://github.com/vdigpku/dada上找到。
Data augmentation (DA) techniques aim to increase data variability, and thus train deep networks with better generalisation. The pioneering AutoAugment automated the search for optimal DA policies with reinforcement learning. However, AutoAugment is extremely computationally expensive, limiting its wide applicability. Followup works such as Population Based Augmentation (PBA) and Fast AutoAugment improved efficiency, but their optimization speed remains a bottleneck. In this paper, we propose Differentiable Automatic Data Augmentation (DADA) which dramatically reduces the cost. DADA relaxes the discrete DA policy selection to a differentiable optimization problem via Gumbel-Softmax. In addition, we introduce an unbiased gradient estimator, RELAX, leading to an efficient and effective one-pass optimization strategy to learn an efficient and accurate DA policy. We conduct extensive experiments on CIFAR-10, CIFAR-100, SVHN, and ImageNet datasets. Furthermore, we demonstrate the value of Auto DA in pre-training for downstream detection problems. Results show our DADA is at least one order of magnitude faster than the state-of-the-art while achieving very comparable accuracy. The code is available at https://github.com/VDIGPKU/DADA.
