论文标题
低延迟峰值神经网络的优化潜在初始化
Optimized Potential Initialization for Low-latency Spiking Neural Networks
论文作者
论文摘要
由于低功耗，生物学合理性和对抗性鲁棒性的独特特性，尖峰神经网络（SNN）非常重要。训练深SNN的最有效方法是通过ANN-SNN转换，在深层网络结构和大规模数据集中产生了最佳性能。但是，准确性和延迟之间存在权衡。为了获得高精度作为原始ANNS，需要长时间的模拟时间来使尖峰神经元的触发速率与模拟神经元的激活值相匹配，这阻碍了SNN的实际应用。在本文中，我们旨在实现具有极低延迟的高性能转换SNN（少于32个时步）。我们从理论上分析ANN-SNN转换开始，并表明缩放阈值的作用与重量归一化的作用相似。我们通过优化初始膜潜力来减少每一层的转换损失，而不是以模型容量为代价引入促进ANN-SNN转换的约束，而是采用了更直接的方式。此外，我们证明了膜电位的最佳初始化可以实现预期的无错误的ANN-SNN转换。我们在CIFAR-10，CIFAR-100和IMAGENET数据集上评估了算法，并使用较少的时间步长可以达到最先进的精度。例如，我们以16个时间步长达到CIFAR-10上的93.38 \％的TOP-1精度。此外，我们的方法可以应用于其他ANN-SNN转换方法，并在时间步长较小时明显促进性能。
Spiking Neural Networks (SNNs) have been attached great importance due to the distinctive properties of low power consumption, biological plausibility, and adversarial robustness. The most effective way to train deep SNNs is through ANN-to-SNN conversion, which have yielded the best performance in deep network structure and large-scale datasets. However, there is a trade-off between accuracy and latency. In order to achieve high precision as original ANNs, a long simulation time is needed to match the firing rate of a spiking neuron with the activation value of an analog neuron, which impedes the practical application of SNN. In this paper, we aim to achieve high-performance converted SNNs with extremely low latency (fewer than 32 time-steps). We start by theoretically analyzing ANN-to-SNN conversion and show that scaling the thresholds does play a similar role as weight normalization. Instead of introducing constraints that facilitate ANN-to-SNN conversion at the cost of model capacity, we applied a more direct way by optimizing the initial membrane potential to reduce the conversion loss in each layer. Besides, we demonstrate that optimal initialization of membrane potentials can implement expected error-free ANN-to-SNN conversion. We evaluate our algorithm on the CIFAR-10, CIFAR-100 and ImageNet datasets and achieve state-of-the-art accuracy, using fewer time-steps. For example, we reach top-1 accuracy of 93.38\% on CIFAR-10 with 16 time-steps. Moreover, our method can be applied to other ANN-SNN conversion methodologies and remarkably promote performance when the time-steps is small.
