论文标题
通过域适应来处理语义细分的新目标类别
Handling new target classes in semantic segmentation with domain adaptation
论文作者
论文摘要
在这项工作中，我们在语义场景分割中定义并解决了新的域适应性（DA）问题，其中目标域不仅显示了数据分布偏移W.R.T.来源领域，但还包括后者不存在的新颖类。与“开放式”和“通用域适应”不同，这两者都将新类别的所有对象视为“未知”，我们旨在明确的这些新类别的测试时间预测。为了实现这一目标，我们提出了一个框架，该框架利用域的适应性和零射击学习技术来实现目标域中的“无限”适应性。它依靠一种新颖的体系结构以及专门的学习方案来弥合源目标域间隙，同时学习如何将新类的标签映射到相关的视觉表示。使用目标域伪标签上的自我训练进一步提高了性能。为了进行验证，我们考虑了不同的域适应设置，即合成-2-Real，Country-2-country和DataSet-2-Dataset。我们的框架的表现优于基准，从而在所有基准上为新任务设定了竞争标准。代码和型号可在https://github.com/valeoai/buda上找到。
In this work, we define and address a novel domain adaptation (DA) problem in semantic scene segmentation, where the target domain not only exhibits a data distribution shift w.r.t. the source domain, but also includes novel classes that do not exist in the latter. Different to "open-set" and "universal domain adaptation", which both regard all objects from new classes as "unknown", we aim at explicit test-time prediction for these new classes. To reach this goal, we propose a framework that leverages domain adaptation and zero-shot learning techniques to enable "boundless" adaptation in the target domain. It relies on a novel architecture, along with a dedicated learning scheme, to bridge the source-target domain gap while learning how to map new classes' labels to relevant visual representations. The performance is further improved using self-training on target-domain pseudo-labels. For validation, we consider different domain adaptation set-ups, namely synthetic-2-real, country-2-country and dataset-2-dataset. Our framework outperforms the baselines by significant margins, setting competitive standards on all benchmarks for the new task. Code and models are available at https://github.com/valeoai/buda.
