论文标题
部分可观测时空混沌系统的无模型预测
ECON: Explicit Clothed humans Optimized via Normal integration
论文作者
论文摘要
深度学习，艺术家策划的扫描和隐式功能（如果）的结合正在使图像中的详细，衣服，3D人类的创建。但是，现有方法远非完美。如果基于方法恢复了自由形式的几何形状，但会为新颖的姿势或衣服产生无形的肢体或归化形状。为了提高这些情况下的鲁棒性，现有工作使用明确的参数身体模型来限制表面重建，但这限制了自由形式表面的恢复，例如偏离人体的松散衣服。我们想要的是一种结合隐式表示的最佳特性和明确的身体正则化的方法。为此，我们进行了两个关键的观察：（1）当前网络比全3D表面更好地推断详细的2D地图，并且（2）可以将参数模型视为用于将详细表面贴片拼接在一起的“帆布”。基于这些，我们的方法Econ具有三个主要步骤：（1）它详细介绍了衣服的前侧的2D正常地图。 （2）从这些过程中，它恢复了2.5D的前后表面，称为D-bini，它们同样详细但不完整，并记录了这些W.R.T.借助从图像中恢复的SMPL-X身体网眼彼此。 （3）它“涂” D-bini表面之间缺少几何形状。如果脸部和手是嘈杂的，则可以选择将它们替换为SMPL-X的。结果，即使是穿着宽松的衣服和挑战性的姿势，Econ也侵犯了高保真3D人类。根据CAPE和RenderPeople数据集的定量评估，这超出了以前的方法。感知研究还表明，经济学的现实主义通过很大的余地更好。代码和模型可用于研究目的，可在econ.is.tue.mpg.de上使用
The combination of deep learning, artist-curated scans, and Implicit Functions (IF), is enabling the creation of detailed, clothed, 3D humans from images. However, existing methods are far from perfect. IF-based methods recover free-form geometry, but produce disembodied limbs or degenerate shapes for novel poses or clothes. To increase robustness for these cases, existing work uses an explicit parametric body model to constrain surface reconstruction, but this limits the recovery of free-form surfaces such as loose clothing that deviates from the body. What we want is a method that combines the best properties of implicit representation and explicit body regularization. To this end, we make two key observations: (1) current networks are better at inferring detailed 2D maps than full-3D surfaces, and (2) a parametric model can be seen as a "canvas" for stitching together detailed surface patches. Based on these, our method, ECON, has three main steps: (1) It infers detailed 2D normal maps for the front and back side of a clothed person. (2) From these, it recovers 2.5D front and back surfaces, called d-BiNI, that are equally detailed, yet incomplete, and registers these w.r.t. each other with the help of a SMPL-X body mesh recovered from the image. (3) It "inpaints" the missing geometry between d-BiNI surfaces. If the face and hands are noisy, they can optionally be replaced with the ones of SMPL-X. As a result, ECON infers high-fidelity 3D humans even in loose clothes and challenging poses. This goes beyond previous methods, according to the quantitative evaluation on the CAPE and Renderpeople datasets. Perceptual studies also show that ECON's perceived realism is better by a large margin. Code and models are available for research purposes at econ.is.tue.mpg.de
