论文标题
Lassie：从稀疏图像合奏中学习清晰的形状，通过3D部分发现
LASSIE: Learning Articulated Shapes from Sparse Image Ensemble via 3D Part Discovery
论文作者
论文摘要
通过手动创建或使用3D扫描工具来创建高质量的铰接3D动物3D模型。因此，从2D图像重建铰接的3D对象的技术至关重要且非常有用。在这项工作中，我们提出了一个实用问题设置，以估计只有几个（10-30）特定动物物种（例如马）的野外图像（Horse）的3D姿势和形状。与依赖于预定义模板形状的现有作品相反，我们不假定任何形式的2D或3D地面真相注释，也不利用任何多视图或时间信息。此外，每个输入图像合奏都可以包含具有不同姿势，背景，照明和纹理的动物实例。我们的主要见解是，与整体动物相比，3D零件的形状要简单得多，并且它们是强大的W.R.T.动物姿势关节。遵循这些见解，我们提出了Lassie，这是一个新颖的优化框架，以最少的用户干预以自我监督的方式发现3D部分。 Lassie背后的关键推动力是使用自我探讨的深度功能实现2D-3D零件的一致性。与先前的艺术相比，关于Pascal-Part和自我收集的野生动物数据集的实验表明，3D重建以及2D和3D部分的发现都更好。项目页面：chhankyao.github.io/lassie/
Creating high-quality articulated 3D models of animals is challenging either via manual creation or using 3D scanning tools. Therefore, techniques to reconstruct articulated 3D objects from 2D images are crucial and highly useful. In this work, we propose a practical problem setting to estimate 3D pose and shape of animals given only a few (10-30) in-the-wild images of a particular animal species (say, horse). Contrary to existing works that rely on pre-defined template shapes, we do not assume any form of 2D or 3D ground-truth annotations, nor do we leverage any multi-view or temporal information. Moreover, each input image ensemble can contain animal instances with varying poses, backgrounds, illuminations, and textures. Our key insight is that 3D parts have much simpler shape compared to the overall animal and that they are robust w.r.t. animal pose articulations. Following these insights, we propose LASSIE, a novel optimization framework which discovers 3D parts in a self-supervised manner with minimal user intervention. A key driving force behind LASSIE is the enforcing of 2D-3D part consistency using self-supervisory deep features. Experiments on Pascal-Part and self-collected in-the-wild animal datasets demonstrate considerably better 3D reconstructions as well as both 2D and 3D part discovery compared to prior arts. Project page: chhankyao.github.io/lassie/
