论文标题
通过分析图像匹配来解释DeepFake检测
Explaining Deepfake Detection by Analysing Image Matching
论文作者
论文摘要
本文旨在解释当刚被二进制标签监督时，深泡检测模型如何学习图像的伪影特征。为此，从图像匹配的角度提出了三个假设，如下所示。 1。深泡检测模型指出了基于既不是源与源相关的视觉概念也不相关的视觉概念，也就是说，考虑到与伪影这样的视觉概念。 2。除了对二进制标签的监督外，DeepFake检测模型还通过训练集中的FST匹配（即匹配的伪造，源，目标图像）隐含地学习与人工制品相关的视觉概念。 3。通过原始训练集中的FST匹配，隐式学习的人工视觉概念很容易受到视频压缩的影响。在实验中，在各种DNN中验证了上述假设。此外，根据这种理解，我们提出了FST匹配的DeepFake检测模型，以提高压缩视频中伪造检测的性能。实验结果表明，我们的方法实现了出色的性能，尤其是在高度压缩的（例如C40）视频上。
This paper aims to interpret how deepfake detection models learn artifact features of images when just supervised by binary labels. To this end, three hypotheses from the perspective of image matching are proposed as follows. 1. Deepfake detection models indicate real/fake images based on visual concepts that are neither source-relevant nor target-relevant, that is, considering such visual concepts as artifact-relevant. 2. Besides the supervision of binary labels, deepfake detection models implicitly learn artifact-relevant visual concepts through the FST-Matching (i.e. the matching fake, source, target images) in the training set. 3. Implicitly learned artifact visual concepts through the FST-Matching in the raw training set are vulnerable to video compression. In experiments, the above hypotheses are verified among various DNNs. Furthermore, based on this understanding, we propose the FST-Matching Deepfake Detection Model to boost the performance of forgery detection on compressed videos. Experiment results show that our method achieves great performance, especially on highly-compressed (e.g. c40) videos.
