论文标题
从测试有效性的角度重新审视深度神经网络测试覆盖范围
Revisiting Deep Neural Network Test Coverage from the Test Effectiveness Perspective
论文作者
论文摘要
已经提出了许多测试覆盖率指标来测量深层神经网络（DNN）测试效果，包括结构覆盖范围和非结构覆盖范围。这些测试覆盖范围是根据基本假设提出的：它们与测试有效性相关。但是，基本假设仍未得到充分和合理的验证，这对DNN测试覆盖率的有用性提出了疑问。本文从测试有效性的角度进行了对现有DNN测试覆盖率的重新审视研究，以有效验证基本假设。在这里，我们仔细考虑了受试者的多样性，三个测试有效性标准，以及典型和最先进的测试覆盖率指标。与所有有关现有DNN测试覆盖范围有用性的负面结论的现有研究不同，我们从测试有效性的角度得出了一些积极的结论。特别是，我们发现了结构性和非结构性覆盖范围之间的互补关系，并确定了这些现有的测试覆盖范围指标的实际用法方案和有前途的研究方向。
Many test coverage metrics have been proposed to measure the Deep Neural Network (DNN) testing effectiveness, including structural coverage and non-structural coverage. These test coverage metrics are proposed based on the fundamental assumption: they are correlated with test effectiveness. However, the fundamental assumption is still not validated sufficiently and reasonably, which brings question on the usefulness of DNN test coverage. This paper conducted a revisiting study on the existing DNN test coverage from the test effectiveness perspective, to effectively validate the fundamental assumption. Here, we carefully considered the diversity of subjects, three test effectiveness criteria, and both typical and state-of-the-art test coverage metrics. Different from all the existing studies that deliver negative conclusions on the usefulness of existing DNN test coverage, we identified some positive conclusions on their usefulness from the test effectiveness perspective. In particular, we found the complementary relationship between structural and non-structural coverage and identified the practical usage scenarios and promising research directions for these existing test coverage metrics.
