论文标题
评估可靠建筑物损害检测的局外概括
Assessing out-of-domain generalization for robust building damage detection
论文作者
论文摘要
限制自然灾害的负面影响的重要一步是灾难发生后的快速损害评估。例如，通过将计算机视觉技术应用于卫星图像，可以自动化构建损坏检测。这样的模型在多域设置中运行：每一次灾难本质上都不同（新的地理位置，独特的情况），并且模型必须强大，以使可用于培训的灾难图像和新事件的图像之间的分布变化。因此，估计现实世界的性能需要一个室外（OOD）测试集。但是，到目前为止，已经在更简单但不现实的分布（IID）测试设置中评估了建筑损坏检测模型。在这里，我们认为未来的工作应该专注于OOD制度。我们评估了两个竞争性损害检测模型的OOD性能，并发现现有的最新模型显示出很大的概括差距：当对训练期间未使用的新灾害评估时，它们的性能下降。此外，IID的性能不能预测OOD性能，这使当前基准测试对现实世界的性能无信息。代码和模型权重可在https://github.com/ecker-lab/robust-bdd上找到。
An important step for limiting the negative impact of natural disasters is rapid damage assessment after a disaster occurred. For instance, building damage detection can be automated by applying computer vision techniques to satellite imagery. Such models operate in a multi-domain setting: every disaster is inherently different (new geolocation, unique circumstances), and models must be robust to a shift in distribution between disaster imagery available for training and the images of the new event. Accordingly, estimating real-world performance requires an out-of-domain (OOD) test set. However, building damage detection models have so far been evaluated mostly in the simpler yet unrealistic in-distribution (IID) test setting. Here we argue that future work should focus on the OOD regime instead. We assess OOD performance of two competitive damage detection models and find that existing state-of-the-art models show a substantial generalization gap: their performance drops when evaluated OOD on new disasters not used during training. Moreover, IID performance is not predictive of OOD performance, rendering current benchmarks uninformative about real-world performance. Code and model weights are available at https://github.com/ecker-lab/robust-bdd.
