论文标题
Tanhexp：轻质神经网络具有高收敛速度的平滑激活功能
TanhExp: A Smooth Activation Function with High Convergence Speed for Lightweight Neural Networks
论文作者
论文摘要
用于实时计算机视觉任务任务的轻量级或移动神经网络所包含的参数少于普通网络，从而导致性能受到限制。在这项工作中，我们提出了一个名为Tanh指数激活函数（TANHEXP）的新型激活函数，该功能可以显着改善这些网络在图像分类任务上的性能。 tanhexp的定义是f（x）= xtanh（e^x）。我们证明了Tanhexp在各种数据集和网络模型上的简单性，效率和鲁棒性，而Tanhexp在收敛速度和准确性方面都优于其对应物。即使添加噪声和数据集更改，它的行为也保持稳定。我们表明，在不增加网络的大小的情况下，Tanhex的轻量级神经网络的能力只需少数训练时期，而没有添加额外的参数。
Lightweight or mobile neural networks used for real-time computer vision tasks contain fewer parameters than normal networks, which lead to a constrained performance. In this work, we proposed a novel activation function named Tanh Exponential Activation Function (TanhExp) which can improve the performance for these networks on image classification task significantly. The definition of TanhExp is f(x) = xtanh(e^x). We demonstrate the simplicity, efficiency, and robustness of TanhExp on various datasets and network models and TanhExp outperforms its counterparts in both convergence speed and accuracy. Its behaviour also remains stable even with noise added and dataset altered. We show that without increasing the size of the network, the capacity of lightweight neural networks can be enhanced by TanhExp with only a few training epochs and no extra parameters added.
