论文标题
3D门控复发融合用于语义场景完成
3D Gated Recurrent Fusion for Semantic Scene Completion
论文作者
论文摘要
本文解决了语义场景完成（SSC）任务中数据融合的问题，该任务可以同时处理语义标签和场景完成。 RGB图像包含对象的纹理细节，这对于语义场景的理解至关重要。同时，深度图像捕获了形状完成的高相关性的几何线索。同时使用RGB和深度图像可以进一步提高SSC在隔离中采用一种模态的准确性。我们提出了一个3D门控复发网络（GRFNET），该网络学会了通过使用门和内存模块来适应从深度和RGB中选择和融合相关信息。基于单级融合，我们进一步提出了一种多阶段融合策略，该策略可以模拟网络内不同阶段之间的相关性。在两个基准数据集上进行的广泛实验证明了拟议的GRFNET在SSC中的数据融合的卓越性能和有效性。代码将提供。
This paper tackles the problem of data fusion in the semantic scene completion (SSC) task, which can simultaneously deal with semantic labeling and scene completion. RGB images contain texture details of the object(s) which are vital for semantic scene understanding. Meanwhile, depth images capture geometric clues of high relevance for shape completion. Using both RGB and depth images can further boost the accuracy of SSC over employing one modality in isolation. We propose a 3D gated recurrent fusion network (GRFNet), which learns to adaptively select and fuse the relevant information from depth and RGB by making use of the gate and memory modules. Based on the single-stage fusion, we further propose a multi-stage fusion strategy, which could model the correlations among different stages within the network. Extensive experiments on two benchmark datasets demonstrate the superior performance and the effectiveness of the proposed GRFNet for data fusion in SSC. Code will be made available.
