论文标题
CNN+RNN深度和基于骨架的动态手势识别
CNN+RNN Depth and Skeleton based Dynamic Hand Gesture Recognition
论文作者
论文摘要
人类活动和手势识别是迅速发展的环境智力领域的重要组成部分，尤其是在协助生活和智能家居中。在本文中，我们建议使用深度和骨架数据相结合两种深度学习技术，卷积神经网络（CNN）和复发性神经网络（RNN）的功能，以使用深度和骨架数据进行自动手势识别。这些类型的数据中的每一个都可以单独使用来训练神经网络以识别手势。虽然先前据报道，RNN在识别骨架信息的每个骨架关节的运动序列方面表现良好，但本研究旨在利用深度数据并应用CNN从深度图像中提取重要的空间信息。串联CNN+RNN共同能够更准确地识别一系列手势。同样，研究了各种类型的融合，以结合骨骼和深度信息，以提取时间空间信息。在动态手势14/28数据集上，总体精度达到85.46％。
Human activity and gesture recognition is an important component of rapidly growing domain of ambient intelligence, in particular in assisting living and smart homes. In this paper, we propose to combine the power of two deep learning techniques, the convolutional neural networks (CNN) and the recurrent neural networks (RNN), for automated hand gesture recognition using both depth and skeleton data. Each of these types of data can be used separately to train neural networks to recognize hand gestures. While RNN were reported previously to perform well in recognition of sequences of movement for each skeleton joint given the skeleton information only, this study aims at utilizing depth data and apply CNN to extract important spatial information from the depth images. Together, the tandem CNN+RNN is capable of recognizing a sequence of gestures more accurately. As well, various types of fusion are studied to combine both the skeleton and depth information in order to extract temporal-spatial information. An overall accuracy of 85.46% is achieved on the dynamic hand gesture-14/28 dataset.
