论文标题
通过自动编码的图像图像图像翻译
Image-Graph-Image Translation via Auto-Encoding
论文作者
论文摘要
这项工作介绍了第一个卷积神经网络，该网络在不需要外部监督的情况下学习了图像到盖翻译任务。获得图像内容的图表表示，其中对象表示为节点及其关系为边缘，这是场景理解中的重要任务。当前的方法遵循一种完全监督的方法，因此需要细致的注释。为了克服这一点，我们是第一个基于完全差异的自动编码器提出一种自我监督的方法，其中瓶颈编码图形的节点和边缘。这种自我监督的方法当前可以将简单的线图编码到图形中，并根据三重态匹配的F1分数获得与完全监督基线的可比结果。除了这些有希望的结果外，我们还为未来的研究提供了一些方向，以了解如何扩展我们的方法以涵盖更复杂的图像。
This work presents the first convolutional neural network that learns an image-to-graph translation task without needing external supervision. Obtaining graph representations of image content, where objects are represented as nodes and their relationships as edges, is an important task in scene understanding. Current approaches follow a fully-supervised approach thereby requiring meticulous annotations. To overcome this, we are the first to present a self-supervised approach based on a fully-differentiable auto-encoder in which the bottleneck encodes the graph's nodes and edges. This self-supervised approach can currently encode simple line drawings into graphs and obtains comparable results to a fully-supervised baseline in terms of F1 score on triplet matching. Besides these promising results, we provide several directions for future research on how our approach can be extended to cover more complex imagery.
