论文标题
智能手机视频中可重新可重新的3D头肖像
Relightable 3D Head Portraits from a Smartphone Video
论文作者
论文摘要
在这项工作中，介绍了一种用于创建可重新可重视的3D肖像的系统。我们的神经管道以一系列帧摄像机捕获的一系列框架（Flash-no Flash序列）捕获。然后，通过结构 - 动作软件和多视图denoising重建的粗点云被用作几何代理。之后，对深度渲染网络进行了训练，以回归构成新观点的密集反照率，正常和环境照明图。有效地，代理几何形状和渲染网络构成了可重新的3D肖像模型，可以从任意观点和任意照明下综合，例如定向光，点光或环境图。该模型与人脸特异性先验的框架序列拟合，该框架可以实现反照率分解的合理性，并以交互式框架速率运行。我们在不同的照明条件下和推断的观点下评估该方法的性能，并与现有的重新定义方法进行比较。
In this work, a system for creating a relightable 3D portrait of a human head is presented. Our neural pipeline operates on a sequence of frames captured by a smartphone camera with the flash blinking (flash-no flash sequence). A coarse point cloud reconstructed via structure-from-motion software and multi-view denoising is then used as a geometric proxy. Afterwards, a deep rendering network is trained to regress dense albedo, normals, and environmental lighting maps for arbitrary new viewpoints. Effectively, the proxy geometry and the rendering network constitute a relightable 3D portrait model, that can be synthesized from an arbitrary viewpoint and under arbitrary lighting, e.g. directional light, point light, or an environment map. The model is fitted to the sequence of frames with human face-specific priors that enforce the plausibility of albedo-lighting decomposition and operates at the interactive frame rate. We evaluate the performance of the method under varying lighting conditions and at the extrapolated viewpoints and compare with existing relighting methods.
