论文标题

重新审视视频超分辨率的时间建模

Revisiting Temporal Modeling for Video Super-resolution

论文作者

Isobe, Takashi, Zhu, Fang, Jia, Xu, Wang, Shengjin

论文摘要

视频超分辨率在监视视频分析和超高定义视频显示中起着重要作用,这在研究和工业社区中引起了很多关注。尽管已经提出了许多基于深度学习的VSR方法,但很难直接比较这些方法,因为不同的损失函数和培训数据集对超分辨率结果有重大影响。在这项工作中,我们仔细研究并比较了三种时间建模方法(具有早期融合的2D CNN,3D CNN,且融合缓慢和复发性神经网络)用于视频超分辨率。我们还提出了一种新型的复发剩余网络(RRN),以进行有效的视频超分辨率,其中使用残留学习来稳定RNN的训练,同时又可以提高超级分辨率的性能。广泛的实验表明,所提出的RRN是高度的计算效率,并且比其他时间建模方法产生时间一致的VSR结果。此外,所提出的方法在几个广泛使用的基准测试基准上实现了最先进的结果。

Video super-resolution plays an important role in surveillance video analysis and ultra-high-definition video display, which has drawn much attention in both the research and industrial communities. Although many deep learning-based VSR methods have been proposed, it is hard to directly compare these methods since the different loss functions and training datasets have a significant impact on the super-resolution results. In this work, we carefully study and compare three temporal modeling methods (2D CNN with early fusion, 3D CNN with slow fusion and Recurrent Neural Network) for video super-resolution. We also propose a novel Recurrent Residual Network (RRN) for efficient video super-resolution, where residual learning is utilized to stabilize the training of RNN and meanwhile to boost the super-resolution performance. Extensive experiments show that the proposed RRN is highly computational efficiency and produces temporal consistent VSR results with finer details than other temporal modeling methods. Besides, the proposed method achieves state-of-the-art results on several widely used benchmarks.

扫码加入交流群

加入微信交流群

微信交流群二维码

扫码加入学术交流群,获取更多资源