论文标题
DeepFaceFlow：野外密集的3D面部运动估计
DeepFaceFlow: In-the-wild Dense 3D Facial Motion Estimation
论文作者
论文摘要
仅来自单眼内对RGB图像的密集的3D面部运动捕获是一个高度挑战的问题，从面部表达识别到面部重演，范围为许多应用。在这项工作中，我们提出了DeepFaceFlow，这是一个强大，快速且高度准确的框架，以对成对的单眼图像之间的3D非刚性面部流进行密集的估计。我们的DeepFaceFlow框架在两个非常大规模的面部视频数据集上进行了训练和测试，其中一个是我们自己的收集和注释，并借助遮挡和基于3D的损失功能。我们进行了全面的实验，以探测方法的各个方面，并证明其针对最先进的流量和3D重建方法的性能提高。此外，我们将我们的框架纳入了最先进的面部视频合成方法中，并演示了我们方法更好地表示和捕获面部动力学的能力，从而产生了高度现实的面部视频综合。给定的注册成对的图像，我们的框架在〜60 fps下生成3D流图。
Dense 3D facial motion capture from only monocular in-the-wild pairs of RGB images is a highly challenging problem with numerous applications, ranging from facial expression recognition to facial reenactment. In this work, we propose DeepFaceFlow, a robust, fast, and highly-accurate framework for the dense estimation of 3D non-rigid facial flow between pairs of monocular images. Our DeepFaceFlow framework was trained and tested on two very large-scale facial video datasets, one of them of our own collection and annotation, with the aid of occlusion-aware and 3D-based loss function. We conduct comprehensive experiments probing different aspects of our approach and demonstrating its improved performance against state-of-the-art flow and 3D reconstruction methods. Furthermore, we incorporate our framework in a full-head state-of-the-art facial video synthesis method and demonstrate the ability of our method in better representing and capturing the facial dynamics, resulting in a highly-realistic facial video synthesis. Given registered pairs of images, our framework generates 3D flow maps at ~60 fps.
