论文标题
生理信号深度学习的注意机制：我们应该引起哪种关注？
Attention mechanisms for physiological signal deep learning: which attention should we take?
论文作者
论文摘要
注意机制被广泛用于大大改善各个领域的深度学习模型性能。但是，他们提高生理信号深度学习模型的性能的一般能力并非成熟。在这项研究中，我们通过实验分析了四种注意机制（例如，挤压和激发，非本地，卷积阻止注意模块和多头自我关注）和三个卷积神经网络（CNN）体系结构（例如VGG，Resnet和Inception）的两种代表性信号预测任务： （CO）。我们评估了多种组合的生理信号深度学习模型的性能和收敛性。因此，具有空间注意机制的CNN模型在分类问题中表现出最佳性能，而通道注意机制在回归问题中达到了最低的误差。此外，在这两个问题中，CNN模型与注意机制的性能和收敛性都比独立的自我注意力模型更好。因此，尽管独立的自我注意力学模型需要更少的参数，但我们验证了卷积操作和注意机制是互补的，并提供了更快的收敛时间。
Attention mechanisms are widely used to dramatically improve deep learning model performance in various fields. However, their general ability to improve the performance of physiological signal deep learning model is immature. In this study, we experimentally analyze four attention mechanisms (e.g., squeeze-and-excitation, non-local, convolutional block attention module, and multi-head self-attention) and three convolutional neural network (CNN) architectures (e.g., VGG, ResNet, and Inception) for two representative physiological signal prediction tasks: the classification for predicting hypotension and the regression for predicting cardiac output (CO). We evaluated multiple combinations for performance and convergence of physiological signal deep learning model. Accordingly, the CNN models with the spatial attention mechanism showed the best performance in the classification problem, whereas the channel attention mechanism achieved the lowest error in the regression problem. Moreover, the performance and convergence of the CNN models with attention mechanisms were better than stand-alone self-attention models in both problems. Hence, we verified that convolutional operation and attention mechanisms are complementary and provide faster convergence time, despite the stand-alone self-attention models requiring fewer parameters.
