论文标题
语义场景综合：应用于辅助系统
Semantic scene synthesis: Application to assistive systems
论文作者
论文摘要
这项工作的目的是从单个深度图像提供语义场景综合。这用于辅助援助系统，用于视觉障碍和盲人，使他们可以通过触摸感理解周围环境。盲人使用触摸来识别物体并依靠倾听来取代视力的事实激励我们提出这项工作。首先，对获得的深度图像进行了细分，并使用深度学习网络在辅助系统的背景下进行分类。其次，受盲文系统和日本写作系统汉字的启发，获得的课程用语义标签编码。然后使用这些标签和提取的几何特征合成场景。我们的系统只能通过了解提供的说明性标签来预测17个类以上的类别。对于其余对象，它们的几何特征将传输。标签和几何特征被映射在触摸感的合成区域。实验是在嘈杂和不完整的数据上进行的，包括室内场景和公共数据集的深度图像。报告并讨论了获得的结果。
The aim of this work is to provide a semantic scene synthesis from a single depth image. This is used in assistive aid systems for visually impaired and blind people that allow them to understand their surroundings by the touch sense. The fact that blind people use touch to recognize objects and rely on listening to replace sight, motivated us to propose this work. First, the acquired depth image is segmented and each segment is classified in the context of assistive systems using a deep learning network. Second, inspired by the Braille system and the Japanese writing system Kanji, the obtained classes are coded with semantic labels. The scene is then synthesized using these labels and the extracted geometric features. Our system is able to predict more than 17 classes only by understanding the provided illustrative labels. For the remaining objects, their geometric features are transmitted. The labels and the geometric features are mapped on a synthesis area to be sensed by the touch sense. Experiments are conducted on noisy and incomplete data including acquired depth images of indoor scenes and public datasets. The obtained results are reported and discussed.
