论文标题
用RGB热图像对分割玻璃分割
Glass Segmentation with RGB-Thermal Image Pairs
论文作者
论文摘要
本文提出了一种使用配对的RGB和热图像的新玻璃分割方法。由于可见光的透射特性与通过玻璃的热能的传输特性之间的差异很大，在该玻璃中，大多数玻璃与可见光透明，但不透明对热能的不透明，因此场景的玻璃区域比仅使用RGB图像更明显地用RGB和热图像来区分。为了利用这种独特的属性，我们提出了一种神经网络结构，该神经网络体系结构有效地将RGB热图像对与基于注意力的新的多模式融合模块结合在一起，并将CNN和Transfers整合以分别提取本地特征和非局部特征。同样，我们已经收集了一个新的数据集，其中包含5551个RGB热图像对，并带有基础真相分割注释。定性和定量评估证明了拟议方法对融合RGB和玻璃分割的热数据的有效性。我们的代码和数据可在https://github.com/dong-huo/rgb-t-glass-vermegnation上获得。
This paper proposes a new glass segmentation method utilizing paired RGB and thermal images. Due to the large difference between the transmission property of visible light and that of the thermal energy through the glass where most glass is transparent to the visible light but opaque to thermal energy, glass regions of a scene are made more distinguishable with a pair of RGB and thermal images than solely with an RGB image. To exploit such a unique property, we propose a neural network architecture that effectively combines an RGB-thermal image pair with a new multi-modal fusion module based on attention, and integrate CNN and transformer to extract local features and non-local dependencies, respectively. As well, we have collected a new dataset containing 5551 RGB-thermal image pairs with ground-truth segmentation annotations. The qualitative and quantitative evaluations demonstrate the effectiveness of the proposed approach on fusing RGB and thermal data for glass segmentation. Our code and data are available at https://github.com/Dong-Huo/RGB-T-Glass-Segmentation.
