论文标题
自我监督语义匹配的信任感的对抗性学习
Confidence-aware Adversarial Learning for Self-supervised Semantic Matching
论文作者
论文摘要
在本文中，我们旨在解决语义匹配的具有挑战性的任务，即使有学习的深度功能，也很难解决匹配的歧义。我们通过考虑对预测的信心并制定新颖的改进策略来纠正部分匹配错误来解决这个问题。具体来说，我们引入了一个信心感知的语义匹配网络（CAMNET），该网络实例化了我们方法的两个关键思想。首先，我们建议通过自我监督的学习来估算一个密集的置信图，以进行匹配的预测。其次，基于估计的置信度，我们通过将可靠的匹配与图像平面上的其余位置传播来完善初始预测。此外，我们开发了一种新的混合损失，在该损失中，我们将语义对准损失与置信度损失相结合，以及衡量语义通信质量的对抗性损失。我们是第一个在完善过程中利用信心以提高语义匹配精度并为整个匹配网络开发端到端自我监管的对抗学习过程的人。我们在两个公共基准上评估了我们的方法，在这两个公共基准测试中，我们在先前的现状中实现了最高的性能。我们将在https://github.com/shuaiyihuang/camnet上发布我们的源代码。
In this paper, we aim to address the challenging task of semantic matching where matching ambiguity is difficult to resolve even with learned deep features. We tackle this problem by taking into account the confidence in predictions and develop a novel refinement strategy to correct partial matching errors. Specifically, we introduce a Confidence-Aware Semantic Matching Network (CAMNet) which instantiates two key ideas of our approach. First, we propose to estimate a dense confidence map for a matching prediction through self-supervised learning. Second, based on the estimated confidence, we refine initial predictions by propagating reliable matching to the rest of locations on the image plane. In addition, we develop a new hybrid loss in which we integrate a semantic alignment loss with a confidence loss, and an adversarial loss that measures the quality of semantic correspondence. We are the first that exploit confidence during refinement to improve semantic matching accuracy and develop an end-to-end self-supervised adversarial learning procedure for the entire matching network. We evaluate our method on two public benchmarks, on which we achieve top performance over the prior state of the art. We will release our source code at https://github.com/ShuaiyiHuang/CAMNet.
