论文标题
变质关系的优先级以降低测试成本
Prioritization of Metamorphic Relations to reduce the cost of testing
论文作者
论文摘要
甲骨文是确定执行测试用例程序的输出是否正确的机制。对于机器学习程序，这种甲骨文是不可用的，也不难用。变质测试是一种使用变质关系的测试方法，它是正在测试的软件的必要属性，以帮助验证程序的正确性。变质关系的优先级有助于降低变质测试的成本[1]。但是，基于代码覆盖范围的变质关系的优先级对于机器学习计划的MRS优先级无效，因为从培训数据中学到了机器学习模型的决策逻辑，并且可以通过单个测试输入轻松实现100％的代码覆盖率。为此，在这项工作中，我们提出了一种基于来源多样性和后续数据集的多样性的成本效益方法，以优先考虑机器学习计划的变质关系。我们表明，与基于代码覆盖的方法相比，提出的基于数据多样性的优先级方法方法将故障检测效率提高了40％，并且与MRS的随机执行相比，将故障检测到的时间降低了29％。总体而言，我们的方法可以节省测试期间的时间和成本。
An oracle is a mechanism to decide whether the outputs of the program for the executed test cases are correct. For machine learning programs, such oracle is not available or too difficult to apply. Metamorphic testing is a testing approach that uses metamorphic relations, which are necessary properties of the software under test to help verify the correctness of a program. Prioritization of metamorphic relations helps to reduce the cost of metamorphic testing [1]. However, prioritizing metamorphic relations based on code coverage is often not effective for prioritizing MRs for machine learning programs, since the decision logic of a machine learning model is learned from training data, and 100% code coverage can be easily achieved with a single test input. To this end, in this work, we propose a cost-effective approach based on diversity in the source and follow-up data set to prioritize metamorphic relations for machine learning programs. We show that the proposed data diversity-based prioritization approach increase the fault detection effectiveness by up to 40% when compared to the code coverage-based approach and reduce the time taken to detect a fault by 29% when compared to random execution of MRs. Overall, our approach leads to saving time and cost during testing.
