论文标题
通过2D-3D-2D周期的语义通信
Semantic Correspondence via 2D-3D-2D Cycle
论文作者
论文摘要
视觉语义通信是计算机视觉中的一个重要主题，可以帮助机器了解我们日常生活中的对象。但是，大多数以前的方法直接训练2D图像中的对应关系，这是端到端，但在3D空间中丢失了大量信息。在本文中，我们提出了一种新方法，通过将其利用到3D域，然后将相应的3D模型换回2D域，并带有其语义标签。我们的方法利用了3D视觉中的优势，可以明确理由关于对象的自我封锁和可见性。我们表明，我们的方法在标准语义基准方面给出了比较甚至优越的结果。我们还进行了详尽而详细的实验，以分析我们的网络组件。代码和实验可在https://github.com/qq456cvb/semantictransfer上公开获得。
Visual semantic correspondence is an important topic in computer vision and could help machine understand objects in our daily life. However, most previous methods directly train on correspondences in 2D images, which is end-to-end but loses plenty of information in 3D spaces. In this paper, we propose a new method on predicting semantic correspondences by leveraging it to 3D domain and then project corresponding 3D models back to 2D domain, with their semantic labels. Our method leverages the advantages in 3D vision and can explicitly reason about objects self-occlusion and visibility. We show that our method gives comparative and even superior results on standard semantic benchmarks. We also conduct thorough and detailed experiments to analyze our network components. The code and experiments are publicly available at https://github.com/qq456cvb/SemanticTransfer.
